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ABSTRACT 


The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center 
(TRAC)  and  the  Modeling,  Virtual  Environments,  and  Simulations  Institute  (MOVES)  at 
the  Naval  Postgraduate  School,  Monterey,  California,  developed  the  Assignment 
Scheduling  Capability  for  UAVs  (ASC-U)  simulation  to  assist  in  the  analysis  of 
unmanned  aerial  vehicle  (UAV)  requirements  for  the  current  and  future  force.  ASC-U 
employs  a  discrete  event  simulation  coupled  with  the  optimization  of  a  linear  objective 
function.  At  regular  intervals,  ASC-U  obtains  an  optimal  solution  to  a  simplified 
problem  that  assigns  available  UAVs  to  missions  that  are  available  or  will  be  available 
within  a  future  time  horizon. 

This  thesis  simultaneously  explores  the  effects  of  26  simulation  and  UAV  factors 
on  the  mission  value  derived  when  allocating  UAVs  to  mission  areas.  The  analysis 
assists  in  defining  the  near  term  (2008)  UAV  force  structure  and  the  investment  strategy 
for  the  mid  term  (2013),  and  far  term  (2018).  We  combine  an  efficient  experimental 
design,  exploratory  modeling,  and  data  analysis  to  examine  514  variations  of  a  scenario 
involving  five  UAV  classes  and  over  21,000  mission  areas.  The  conclusions  suggest  the 
following:  the  optimization  interval  significantly  influences  the  quality  of  the  solution, 
percent  mission  coverage  may  depend  on  a  few  UAV  performance  factors,  small  time 
horizons  increase  percent  mission  coverage,  and  carefully  planned  designs  assist  in  the 
exploration  of  the  outer  and  interior  regions  of  the  response  surface. 
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EXECUTIVE  SUMMARY 


The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center 
(TRAC)  and  the  Modeling,  Virtual  Environments,  and  Simulations  Institute  (MOVES)  at 
the  Naval  Postgraduate  School,  Monterey,  California  developed  the  Assignment 
Scheduling  Capability  for  UAVs  (ASC-U)  simulation  tool  to  assist  in  the  analysis  of 
UAV  requirements.  TRAC  selected  ASC-U  for  the  Army-wide  UAV  Mix  Analysis  that 
supports  investment  strategies  involving  technology  for  current  operations,  transitioning 
to  modular  forces,  and  developing  and  fielding  the  Future  Combat  Systems  (FCS).  The 
purpose  of  this  thesis  is  to  assess  how  changes  in  the  input  factors  concerning  simulation 
settings  (time  horizon  and  optimization  interval)  and  UAV  characteristics  influence  the 
Measures  of  Effectiveness  (MOEs)  generated  by  ASC-U. 

This  thesis  explores  the  effects  of  26  simulation  and  UAV  factors  on  the  MOEs 
generated  by  ASC-U.  We  focus  on  four  specific  areas  of  research  concerning  the 
allocation  of  UAVs  within  the  ASC-U  simulation: 

1 .  What  are  the  appropriate  time  horizons  and  optimization  interval  (decision  point 
where  UAVs  are  allocated  to  mission  areas)  to  use  within  ASC-U? 

2.  Which  UAV  characteristics  and  performance  capabilities  significantly  influence 
the  MOEs? 

3.  Are  there  any  significant  interactions  between  factors  or  relationships  between  the 
MOEs? 

4.  What  are  appropriate  Design  of  Experiment  (DOE)  methodologies  and  tools  to 
incorporate  into  ASC-U  for  the  Army’s  UAV  Mix  Analysis? 

ASC-U  is  a  scheduling  model  that  applies  UAV  resources  against  the  schedule  of 
mission  requirements  within  an  operational  scenario.  ASC-U  employs  a  discrete  event 
simulation  coupled  with  the  optimization  of  a  linear  objective  function.  The  model  is 
based  on  the  framework  developed  for  Dynamic  Allocation  of  Fires  and  Sensors  used  to 
evaluate  factors  associated  with  networking  assets  in  the  FCS.  Given  a  scenario  and  a 
mix  of  UAVs,  ASC-U  determines  a  schedule  for  executing  UAV  missions.  The 
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simulation  tool  considers  airframe  capabilities,  payloads,  the  locations  and  capacities  of 
ground  control  stations  and  launch  and  recovery  sites,  remote  viewing  terminals 
requirements,  and  communication  footprints  and. 

The  scenario  consists  of  a  set  of  mission  area  locations  and  requirements,  friendly 
unit  locations,  platform  and  payload  characteristics,  and  ground  control  characteristics. 
Each  mission  area  has  a  set  of  sensor,  weapon,  or  communication  requirements.  The 
mission  areas  do  not  move  during  the  simulation.  Rather,  each  area  has  a  start  and  end 
time  associated  with  it.  As  the  simulation  proceeds,  mission  areas  “open”  and  “close” 
creating  a  time  window  in  which  UAVs  with  correct  capabilities  can  service  the  mission. 
The  number  and  type  of  UAYs  constrain  the  solution,  as  do  UAV  performance 
capabilities  such  as  air  speed,  operating  time,  transition  time,  and  operating  radius.  The 
number  and  capacities  of  ground  control  stations  (GCSs)  and  launch  and  recovery  sites 
(LRSs)  also  constrain  the  solution.  Additionally,  the  GCSs  and  LRSs  move  during  the 
execution  of  the  scenario. 

The  research  conducted  in  this  thesis  takes  advantage  of  experimental  design 
techniques  based  on  statistical  theory  and  developed  to  assist  researchers  in  the  analysis 
of  computer  simulations.  The  primary  goal  of  experimental  design  is  to  assess  how 
changes  in  input  parameters  (factors)  affect  the  results  (responses).  We  select  26  factors 
for  our  exploration  consisting  of  25  UAV  performance  characteristics  and  the 
simulation’s  optimization  interval.  The  ranges  for  the  UAV  factors  are  created  by  adding 
and  subtracting  20%  of  the  base  scenario  value.  We  set  the  range  for  the  optimization 
interval  between  one  and  ten  simulation  hours. 

We  use  the  space-filling  design  of  Nearly  Orthogonal  Latin  Hypercubes  (NOLH) 
to  provide  an  exploration  of  the  outer  and  interior  regions  of  the  response  surface.  NOLH 
designs  allow  the  analyst  to  develop  a  comprehensive  set  of  explanatory  variables 
represented  in  the  model.  Instead  of  being  restricted  to  two  or  three  levels,  the  analyst 
can  create  a  design  that  uses  multiple  or  even  a  continuous  range  of  values  for  each 
factor.  This  facilitates  the  identification  of  non-linearities  within  the  response  surface. 
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In  addition  to  good  space-filling  properties,  orthogonality  is  another  feature  of 
designs  that  make  them  effective.  Orthogonal  designs  have  no  linear  relationship 
between  the  regressors.  The  NOLH  technique  reduces  the  correlation  between  factor 
columns,  creating  a  nearly  orthogonal  design  matrix.  We  can  examine  the  off-diagonal 
elements  within  the  correlation  matrix  of  the  design  in  order  to  measure  the  level  of 
orthogonality.  The  spreadsheet  design  tool  used  in  the  research  has  several  features  that 
guard  against  multicollinearity  and  produce  a  robust  space-filling  design. 

We  combine  data  analysis,  exploratory  modeling  and  a  Nearly  Orthogonal  Latin 
Hypercube  design  to  examine  the  output  of  514  distinct  scenarios  (called  design  points). 
In  doing  so,  we  expand  the  use  of  ASC-U  from  running  single  scenarios  with  baseline 
settings  to  a  full  exploration  across  multiple  factors.  The  application  and  insights  gained 
will  support  current  and  future  warfighters.  Additionally,  the  research  will  assist  in  ASC¬ 
II’ s  development  and  application  across  a  robust  set  of  future  mission  scenarios.  The 
conclusions  suggest  the  following: 

•  The  optimization  interval  significantly  influences  the  UAV  schedule  and  its 

quality. 

•  Mission  value  and  percent  coverage  are  not  equivalent  MOEs. 

•  An  optimization  interval  of  one  simulation  hour  provides  better  correlation 

between  mission  value  and  percent  mission  coverage. 

•  Longer  time  horizons  result  in  poor  allocation  decisions,  as  observed  in  the 

coefficients  of  the  regression  models. 

•  Each  measure  of  effectiveness  has  a  different  set  of  significant  factors. 

•  Class  I  UAV  operating  time  and  Class  IV  operating  radius  are  the  main 

factors  influencing  14  of  21  MOEs. 

•  Class  IV  UAV  operating  radius  begins  to  make  significant  improvement  to 

coverage  at  below  baseline  settings. 

•  Longer  Class  I  UAV  operating  times  increase  all  other  UAV  mission  percent 

coverages. 

•  State-of-the-art  design  of  experiment  and  data  mining  techniques  yield 

insights  impossible  to  gain  otherwise. 
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I. 


INTRODUCTION 


A.  BACKGROUND 

During  the  past  three  years,  the  United  States  Army  has  developed  three 
investment  paths  associated  with  equipping  the  Army.  The  paths  include  investments  in 
the  following:  technology  required  for  the  immediate  needs  of  current  operations, 
transition  to  modular  forces,  and  development  and  fielding  of  the  Future  Combat  System 
(FCS)  equipped  forces.  All  of  these  investment  paths  include  Unmanned  Aerial  Vehicles 
(UAVs),  UAV  organizations,  UAV  payloads,  ground  control  capabilities,  and 
sustainment  (Witsken,  2004). 

The  Army’s  goal  is  to  transition  from  the  current  modular  force  UAV  mix  to  the 
future  force  UAV  mix.  The  analysis  supporting  the  evolution  of  the  modular  force  will 
generate  the  key  tasks  and  purpose  that  involve  a  future  UAV  capability,  and  recommend 
types  and  quantities  of  UAVs  required  by  all  types  of  maneuver  and  support  brigades. 
(Witsken,  2004)  The  Modeling,  Virtual  Environments,  and  Simulations  Institute 
(MOVES)  and  the  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis 
Center  (TRAC)  at  the  Naval  Postgraduate  School  developed  the  Assignment  Scheduling 
Capability  for  UAVs  (ASC-U)  simulation  tool  to  assist  in  the  analysis  of  UAV 
requirements  for  the  modular  force. 

ASC-U  employs  a  discrete  event  simulation  model  coupled  with  the  optimization 
of  a  linear  objective  function.  ASC-U  determines  a  schedule  for  UAV  missions  that  can 
be  successfully  executed  in  a  scenario  with  a  specific  mix  of  UAVs.  At  regular 
intervals,  ASC-U  obtains  an  optimal  solution  to  a  simplified  problem  that  assigns 
available  UAVs  to  missions  that  are  available  or  will  be  available  within  a  future  time 
horizon.  The  simulation  derives  an  overall  mission  value  for  each  assignment  by 
considering  the  required  flight  time  to  each  mission  area  and  the  amount  of  time  each 
UAV  covers  the  area. 

The  Director  of  TRAC  wants  to  compare  and  correlate  the  methodology  and 
results  of  ASC-U  to  other  simulation  programs  such  as  Vector- In-Commander  (VIC), 
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Advanced  Warfighting  Simulation  (AWARS),  Combined  Arms  and  Support  Task  Force 
Evaluation  Model  (CASTFOREM),  and  Combat  XXI.  The  objective  is  to  assess  the 
compatibility  between  all  five  models  to  ensure  accurate  UAV  evaluation  for  future  UAV 
studies  (Witsken,  2004). 


B.  PURPOSE 

The  purpose  of  this  thesis  is  to  assess  how  changes  in  the  input  factors  concerning 
simulation  settings  (time  horizon  and  optimization  interval)  and  UAV  characteristics 
influence  the  overall  mission  value  and  other  MOEs  derived  when  allocating  UAVs  to 
mission  areas  in  the  ASC-U  simulation.  Additionally,  the  analysis  assists  in  defining  the 
near  term  (2008)  UAV  force  structure  and  the  follow-on  development  of  the  investment 
strategy  for  the  mid  term  (2013),  and  far  term  (2018)  (Witsken,  2004). 

C.  RESEARCH  QUESTIONS 

This  thesis  focuses  on  four  specific  areas  of  research  concerning  the  allocation  of 
UAVs  within  the  ASC-U  simulation: 

1.  What  are  the  appropriate  time  horizons  and  optimization  interval  to  use  within 
ASC-U? 

2.  Which  UAV  characteristics  and  performance  capabilities  significantly  influence 
the  MOEs? 

3.  Are  there  any  significant  interactions  between  factors  or  relationships  between  the 
MOEs? 

4.  What  are  appropriate  Design  of  Experiment  (DOE)  methodologies  and  tools  to  for 
the  Army’s  UAV  Mix  Analysis? 

D.  RESEARCH  SCOPE 

The  Assignment  Scheduling  Capability  for  UAVs  (ASC-U)  simulation  tool 
considers  a  dynamic  UAV  routing  problem.  It  determines  a  feasible  schedule  for  UAV 
missions  given  a  scenario  and  a  specific  mix,  type  and  number  of  UAVs.  ASC-U  does 
not  consider  several  factors  that  may  influence  the  solution,  such  as  attrition  of  aircraft, 
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maintenance  and  sustainability,  communications,  terrain  effects  and  weather.  We  do  not 
consider  these  factors  in  this  study.  Our  research  is  confined  to  investigating  the  factors 
considered  by  ASC-U  that  influence  the  allocation  of  UAVs  to  mission  areas.  The 
following  areas  provide  the  scope  of  our  research: 

•  Define  the  capabilities  of  the  simulation. 

•  Provide  a  DOE  methodology  for  the  simulation. 

•  Identify  an  appropriate  optimization  interval. 

•  Identify  appropriate  time  horizons  for  UAV  types. 

•  Assess  the  influence  of  factors  on  the  MOEs  supported  by  ASC-U. 

•  Conduct  data  analysis  on  the  simulation  output. 

•  Fit  regression  models  to  the  data  output. 

E.  THESIS  ORGANIZATION 

Chapter  II  introduces  the  Army’s  Future  Combat  System  program.  It  provides  an 
overview  of  the  types  of  UAVs  considered  in  the  study.  Included  is  a  brief  description  of 
each  aircraft  and  its  capabilities.  Chapter  III  discusses  the  formulation  of  the  UAV 
allocation  problem  and  describes  ASC-U’s  approach  to  the  solution.  Chapter  IV 
describes  the  selection  of  factors  included  in  our  experiment,  the  development  of  design 
points  (scenarios)  on  which  ASC-U  is  tested,  and  the  statistical  software  selected  for  the 
analysis.  Chapter  V  provides  an  analysis  of  the  ASC-U  output  data  and  the  regression 
models  fit  for  21  different  MOEs.  Chapter  VI  summarizes  conclusions  drawn  from  this 
overall  study  and  provides  recommendations  for  follow-on  research. 
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II.  FUTURE  COMBAT  SYSTEM 


It  shall  be  a  goal  of  the  Armed  Forces  to  achieve  the  fielding  of 
unmanned,  remotely  controlled  technology  such  that  by  2010,  one-third  of 
the  operational  deep  strike  aircraft  of  the  Armed  Forces  are  unmanned; 
and  by  2015,  one-third  of  the  operational  ground  combat  vehicles  of  the 
Armed  Forces  are  unmanned  (106th  Congress,  2000). 

National  Defense  Authorization  Act  for 
Fiscal  Year  2001  H.R.4205,  Sec.  217 

The  Army’s  Future  Combat  System  (FCS)  comprises  a  research,  development, 
and  acquisition  program  that  extends  over  three  decades  and  includes  a  multi-billion- 
dollar  budget.  The  Chief  of  Staff  of  the  Army  (CSA)  General  Eric  Shinseki  introduced 
the  FCS  program  in  October  1999.  The  program  originally  entailed  the  transformation  of 
the  Army’s  Legacy  Forces  that  is  comprised  of  divisions  into  a  lighter,  modular 
organization  called  the  Objective  Force.  Initial  plans  were  to  field  the  first  of  such 
modular-equipped  forces  in  2011  and  complete  the  entire  Objective  Force  by  2032.  In 
order  to  bridge  the  gap  between  the  Legacy  Force  and  the  Objective  Force,  and  fill  near- 
term  warfighting  requirements,  the  program  called  for  an  Interim  Force  consisting  of 
active  Army  and  Army  National  Guard  units.  These  brigade-sized  units  are  known  as 
Interim  Brigade  Combat  Teams  (IBCTs)  or  Stryker  Brigade  Combat  Teams  (SBCTs). 
The  first  of  such  units  have  seen  combat  in  Iraq.  The  last  IBCT  is  scheduled  for  fielding 
in  2010  (“Unmanned  Aircraft,”  2005). 

General  Shinseki ’s  successor,  General  Peter  Schoomaker,  changed  portions  of  the 
FCS  program  in  2003.  He  started  by  redefining  the  Objective  Force  as  the  Future  Force 
and  called  for  spiral  development,  incorporating  functional  FCS  capabilities  as  they 
became  available.  Additionally,  General  Schoomaker  placed  greater  importance  on  the 
system  of  networks  required  to  link  Army  forces  as  well  as  Joint  forces  together 
(“Unmanned  Aircraft,”  2005). 
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On  July  21,  2004,  the  Army  announced  another  round  of  restructuring  for  the  FCS 
program.  A  primary  objective  of  the  restructuring  included  four  phases  of  spiraling  in 
new  technologies  to  the  existing  force  starting  in  fiscal  year  2008.  The  remaining  three 
spiral  dates  are  scheduled  to  occur  in  2010,  2012,  and  2014.  The  second  modular- 
equipped  brigade,  also  known  as  a  Unit  of  Action  (UA),  is  scheduled  for  fielding  in  2015 
with  two  more  brigades  fielded  each  year  after  2015  for  a  total  of  15  FCS-equipped 
brigades  (“Unmanned  Aircraft,”  2005). 

The  FCS  is  a  joint  networked  “system  of  systems.  ”  The  intent  is  to  facilitate 
connectivity  with  other  services,  increase  situational  awareness,  and  allow  for  the 
synchronization  of  operations.  The  heart  of  the  FCS  relies  on  advanced  technologies  and 
an  extensive  communications  network.  Binding  this  network  together  are  eighteen 
projected  manned  and  unmanned  systems.  The  net-centric  dimension  of  the  FCS  is 
essential  to  the  Future  Force’s  capability  to  “ see  first,  understand  first,  act  first,  and 
finish  decisively  across  the  full  spectrum  of  operations ”  (Knarr,  Haskins,  &  Mouras, 
2001).  Unmanned  aerial  vehicles  are  central  to  seeing  and  understanding  the  enemy  first. 

The  ASC-U  simulation  tool  is  capable  of  modeling  different  scenarios  with 
potentially  unlimited  varieties  of  UAVs.  In  this  study,  we  consider  several  types  of 
UAYs  based  on  characteristics  and  performance  capabilities  of  the  Interim  Force  UAVs. 
These  include  the  small  (SUAV)  and  tactical  (TUAV)  vehicles,  and  the  extended  range 
multipurpose  (ERMP)  UAV.  We  also  considered  UAVs  that  will  make  up  the  modular 
brigades  found  in  the  Future  Force.  The  following  sections  provide  an  overview  of  the 
different  UAVs  considered  in  the  study.  Included  is  a  brief  description  of  each  aircraft 
and  its  capabilities. 
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A. 


SMALL  UNMANNED  AERIAL  VEHICLE  (SUAV) 


Figure  1.  SUAV  -  RQ-1 1  Raven  (From  “Raven  UAVs,”  2005). 


In  the  1990s,  the  Army  became  interested  in  UAVs  capable  of  providing 
Reconnaissance,  Surveillance  and  Target  Acquisition  (RSTA)  at  the  company  and 
platoon  level.  This  capability  requirement  was  reflected  in  the  FCS  program  with  the  call 
for  Class  I  and  II  UAVs  beginning  in  1999.  The  need  for  small  unmanned  aerial  vehicles 
(SUAVs)  increased  as  forces  were  deployed  in  Operation  Enduring  Freedom  (OEF)  in 
2001  and  again  for  Operation  Iraqi  Freedom  (OIF)  in  2003.  Within  20  weeks  of  the 
authorization  of  program  funding,  the  first  of  these  lightweight  vehicles  were  deployed 
with  forces  in  OEF.  Special  Operations  Forces  and  Rangers  took  advantage  of  the 
SUAVs  in  the  rugged  terrain  of  Afghanistan  (Jenkins  &  Snodgrass,  2005).  The  Army 
purchased  over  500  SUAVs  for  OEF  and  OIF  (“Unmanned  Aircraft,”  2005). 
Commanders  have  been  lauding  their  capabilities  ever  since. 

What  started  this  “rave”  was  a  commercial  off  the  shelf  (COTS)  device  called  the 
Raven,  displayed  in  Figure  1.  The  RQ-1 1A  Raven  is  considered  a  small  unmanned  aerial 
vehicle  (SUAV).  It  provides  intelligence,  reconnaissance,  and  surveillance  (ISR) 
capabilities  to  platoons  through  battalions  in  a  variety  of  scenarios  to  include  Military 
Operations  in  Urban  Terrain  (MOUT)  (Jenkins  &  Snodgrass,  2005).  AeroVironment  Inc. 
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developed  the  Raven  after  its  success  with  the  FQM-151  Pointer  that  has  deployed  with 
the  Marines,  Army,  and  Air  force  since  1988  (“Unmanned  Aircraft,”  2005).  Taking 
advantage  of  technology  developments  in  batteries  and  microelectronics,  the  Raven  is 
two-thirds  the  size  and  weight  of  its  predecessor. 

Table  1  displays  the  performance  capabilities  of  the  RQ-11B  Raven.  The  Raven 
system  consists  of  three  airframes  and  one  Ground  Control  Station  (GCS).  The  platform 
runs  on  single-use  and  rechargeable  batteries  and  is  able  to  employ  three  sets  of  sensor 
packages:  high-resolution  day  camera,  high-resolution  night  imager,  and  a  side-view 
thermal  imager  (“Unmanned  Aerial,”  2004).  Crews  hand-launch  the  Raven  and  recover 
it  using  an  onboard  autonomous  pilot  system  (Jenkins  &  Snodgrass,  2005). 

The  Raven  is  a  combat  multiplier,  providing  units  with  beyond-line-of-sight 
(BLOS)  capability  and  reduction  of  “troops  to  task”  (TTT)  by  keeping  soldiers  out  of 
harm’s  way  when  possible.  The  system  transmits  live  video  images,  along  with  telemetry 
data  such  as  compass  headings  and  location,  to  ground  control  units  (GCUs)  and  remote 
video  terminals  (RVTs)  (“Unmanned  Aircraft,”  2005). 


RQ-1  IB 

RQ-1  IB 

Length 

36  in 

Wing  Span 

55  in 

Gross 

Weight 

4.0  lbs 

Payload 

Weight 

6.5  oz 

Engine 

direct  drive 
electric 

Battery 

LiS02 

Li-Ion 

Endurance 

90  min 

Max  Speeds 

30  mph 

Ceiling 

1,000  ft 

Operating 

Radius 

10  km 

Launch 

hand 

Landing 

auto 

deep  stall 

Sensor 

EO/IR 

Table  1 .  Raven  RQ-1  IB  Characteristics  and  Performance  Data  (After  “Unmanned 

Aircraft,”  2005). 
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B.  TACTICAL  UNMANNED  AERIAL  VEHICLE  (TUAV) 


Figure  2.  TUAV  -  RQ-7  Shadow  (From  “ERMP,”  2005). 


In  December  1999,  AAI  Corporation,  a  subsidiary  of  United  Industrial 
Corporation,  was  awarded  an  initial  contract  to  produce  a  tactical  unmanned  aerial 
vehicle  (TUAV)  system.  The  system  is  designed  to  provide  brigade  commanders  with  a 
host  of  battle  management  capabilities  to  include  target  acquisition  and  battle  damage 
assessment  (BDA)  (“U.S.  Army,”  2001).  Thirty-three  months  later,  the  Assistant 
Secretary  of  the  Army  for  Acquisition,  Logistics  and  Technology,  Claude  Bolton  Jr., 
authorized  the  Shadow  200  RQ-7  TUAV  program  to  enter  into  full  rate  production.  The 
contract  with  AAI  Corporation  is  scheduled  to  provide  41  Shadow  200  TUAVs  for  the 
Army’s  six  Stryker  Brigade  Combat  Teams.  All  six  brigades  will  receive  the  Shadow 
system  by  May  2006  (Mahnken,  2002).  AAI’s  contract  was  a  key  success  for  the  Army’s 
Transformation  program,  demonstrating  an  accelerated  effort  to  fill  a  capability  gap  for 
field  commanders. 

The  Shadow,  displayed  in  Figure  2,  is  intended  for  ground  maneuver  brigade 
commanders.  It  provides  a  sustainable  capability  for  reconnaissance,  surveillance,  target 
acquisition  and  battle  damage  assessment.  The  aerial  platform  transmits  images  and 
telemetry  data  to  control  stations,  providing  the  commander  and  his  staff  with  valuable 
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information  to  plan,  execute,  and  support  ongoing  missions.  The  TUAV’s  platoon 
consists  of  four  airframes,  six  High  Mobility  Multi-purpose  Wheeled  Vehicles 
(HMMWV),  two  Ground  Control  Stations  (GCS),  four  RVTs,  antennas,  and  one  Portable 
Ground  Control  Station  (PGCS)  and  Data  Terminal  (PGDT)  (Mahnken,  2002).  The 
system  includes  three  Shadow  platforms  with  a  fourth  vehicle  issued  to  the  maintenance 
section  (“AAI,”  2006).  The  platoon  consists  of  22  personnel  capable  of  sustaining  flight 
operations  on  a  24-hour  basis  (“Unmanned  Aerial,”  2004). 

Table  2  provides  the  performance  capabilities  of  the  RQ-7A  Shadow.  The  TUAV 
has  a  wingspan  of  approximately  13  feet  and  has  a  gross  weight  of  slightly  over  300 
pounds.  It  can  carry  a  payload  of  60  pounds  and  is  equipped  with  an  electro- 
optic/infrared  sensor  package.  The  platform  is  launched  using  a  pneumatic  launcher  and 
is  recovered  using  an  automatic  landing  system.  During  flight,  control  of  the  Shadow  can 
be  transferred  between  GCUs;  it  can  land  at  alternate  LRSs  to  keep  up  with  a  brigade’s 
rapid  operational  tempo  (OPTEMPO)  (Mahnken,  2002).  The  Shadow  is  capable  of 
loitering  above  a  mission  area  for  four  hours,  at  50  kilometers  from  an  LRS.  It  has  a 
maximum  range  of  125  kilometers,  operating  at  8,000  to  10,000  feet  during  the  day  and 
6,000  to  8,000  feet  at  night  (Mahnken,  2002). 

The  Army’s  deputy  chief  of  staff  G-3  directed  that  the  Shadow  2000  be  fielded 
with  every  maneuver  brigade  in  OIF  and  OEF.  As  of  June  2004,  Shadow  platoons 
existed  in  five  divisions  and  two  Stryker  brigades  (2nd  Infantry  Division  and  25th  Infantry 
Division)  (“Unmanned  Aerial,”  2004). 
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RQ-7A 

RQ-7B 

RQ-7A 

RQ-7B 

Length 

11.2  ft 

11.2  ft 

Wing  Span 

12.8  ft 

14  ft 

Gross 

Weight 

327  1b 

375  lb 

Payload 

Capacity 

601b 

60  lb 

Fuel 

Capacity 

51  lb 

73  lb 

Fuel  Type 

MOGAS 

MOGAS 

Engine 

Make 

UEL 

AR-741 

UEL 

AR-741 

Power 

38  hp 

38  hp 

Data  Link(s) 

LOS  C2 
LOS  Video 

LOS  C2 
LOS  Video 

Frequency 

S-band 

UHF 

C-band 

S-band 

UHF 

C-band 

Endurance 

5  hr 

7  hr 

Max/Loiter 

Speeds 

1 10/70  kt 

105/60  kt 

Ceiling 

14,000  ft 

15,000  ft 

Operating 

Radius 

68  nm 

68  nm 

Launch 

catapult 

catapult 

Landing 

arresting 

wire 

arresting 

wire 

Sensor 

EO/IR 

EO/IR 

Sensor 

Make 

Tamam 
POP  200 

Tamam 
POP  300 

Table  2.  Shadow  RQ-7A/B  Characteristics  and  Performance  Data  (After 

“Unmanned  Aircraft,”  2005). 
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c. 


EXTENDED  RANGE  MULTIPURPOSE  (ERMP)  UAV 


Figure  3.  ERMP  -  Warrior  (From  “ERMP,”  2005). 


The  first  endurance  platform  was  the  General  Atomics  RQ-1  Predator  (Goebel, 
2006).  The  “R”  in  the  nomenclature  indicates  reconnaissance  and  the  “Q”  stands  for  the 
unmanned  nature  of  the  vehicle.  The  Predator  first  saw  combat  in  Bosnia  in  1995  and 
again  in  1999  during  Operation  Allied  Force  (Kosovo).  The  UAV  distinguished  itself  in 
ISR  operations  in  Kosovo  by  providing  video  feeds  to  the  command  center  at  Aviano 
Airbase,  Italy.  Forward  air  controllers  (FACs)  used  the  information  to  acquire  difficult 
targets.  By  the  end  of  the  conflict,  the  UAV  was  upgraded  with  laser  designators  (Bone 
&  Bolkcom,  2003). 

As  the  capabilities  and  sensor  packages  on  the  aircraft  increased  (strike  and  ISR), 
the  nomenclature  was  changed  to  “M”  for  multi-mission  (John,  Shaver,  Lynch, 
Amouzegar,  &  Snyder,  2005).  With  its  new  designation  “MQ,”  the  Predator  was  able  to 
demonstrate  its  new  capabilities  starting  in  OEF  (Bone  &  Bolkcom,  2003).  The  arched 
front  of  the  aircraft  contains  a  forward-looking  Synthetic  Aperture  Radar  (SAR)  that 
scans  from  the  “chin”  panel.  Westinghouse  originally  developed  SAR  for  the  cancelled 
US  Navy  A-12  strike  aircraft.  It  provides  30-centimeter  (1  foot)  resolution  at  operational 
altitudes  of  50,000  feet.  The  Versatron  Skyball  turret  houses  the  electro-optic  (EO)  and 
infrared  (IR)  sensors  (Goebel,  2006).  The  Predator  is  capable  of  carrying  a  laser 
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designator  (LD)  to  illuminate  targets  in  order  to  guide  munitions  launched  by  itself  or 
other  strike  aircraft  to  their  destination.  The  Predator  uses  direct  radio  frequencies  (RF) 
or  satellite  links  to  receive  and  transmit  control  information  and  download  data.  The 
UAV  provides  real-time  data  to  other  platforms,  such  as  the  J-STARS  surveillance 
aircraft,  ground  stations,  and  naval  vessels  (Goebel,  2006). 

In  the  summer  of  2005,  the  U.S.  Army  created  an  Extended  Range  Multi-Purpose 
(ERMP)  UAV  requirement.  This  initiative  began  with  a  contract  to  General  Atomics  for 
an  enhanced  Predator  called  the  Warrior,  displayed  in  Figure  3.  The  improved  endurance 
aircraft  will  boast  a  heavy  communication  relay  capability,  enhanced  strike  payloads,  and 
up  to  thirty-six  hours  of  operating  time  (Bone  &  Bolkcom,  2003).  Additional  payloads 
include  electro-optic/infrared  (EO/IR)  with  laser  designator  and  synthetic  aperture 
radar/moving  target  indicator  (SAR/MTI)  (“Unmanned  Aerial,”  2004).  The  Army  is 
planning  to  purchase  11  systems  consisting  of  12  UAVs  and  5  control  stations  per 
system.  The  initial  fielding  is  projected  to  begin  in  2009  (Bone  &  Bolkcom,  2003). 
Table  3  provides  the  performance  capabilities  of  the  MQ-9A  Predator. 


MQ-9A 

MQ-9A 

Length 

36  ft 

Wing  Span 

66  ft 

Gross  Weight 

10,500  lb 

Payload  Capacity 

*750  lb 

Fuel  Capacity 

4,000  lb 

Fuel  Type 

JP 

Engine  Make 

Honeywell  TPE 
331-10 

Power 

900  shp 

Data  Link(s) 

BLOS 

LOS 

Frequency 

Ku-band 

C-band 

Endurance 

30  hr  clean/ 
16-20  hr  external 

stores 

Max/Loiter 

Speeds 

225/TBD  kt 

Ceiling 

50,000  ft 

Operating  Radius 

2,000  nm 

Launch 

runway 

Landing 

runway 

Sensor 

EO/IR 

Sensor  Make 

MTS-B 

SAR/MTI 

Weapons 

4,  500  lb  class  or 
8-10,  250  lb  class 

*Up  to  3,000  lb  total  externally  on  wing  hardpoints. 

Table  3.  Predator  MQ-9A  Characteristics  and  Performance  Data  (After  “Unmanned 

Aircraft,”  2005). 
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D.  FUTURE  COMBAT  SYSTEM  UAVS 

The  Army’s  concept  of  the  FCS  relies  heavily  on  a  communications  network 
linking  the  system  together.  Unmanned  vehicles  are  an  integral  component  of  the 
network  and  systems-of-systems  assembly.  The  Army  envisions  unmanned  aerial 
vehicles  assigned  to  maneuver  and  support  elements  that  include  infantry,  armor,  scout, 
intelligence,  aviation,  artillery,  and  medical  units.  Mission  capabilities  include 
intelligence,  surveillance  and  reconnaissance  (ISR),  battle  damage  assessment,  targeting, 
convoy  protection,  chemical  agent  and  improvised  explosive  device  (IED)  detection 
(“Unmanned  Aerial,”  2004).  Integration  and  commonality  across  UAV  systems  will  be 
essential  for  seamless  and  effective  operations. 

Each  unit  of  action  (UA)  is  projected  to  have  approximately  200  UAVs  that  are 
organic  across  each  echelon  of  the  command  from  the  platoon  to  brigade  level  (Erwin, 
2003).  The  Army  has  defined  four  classes  for  the  Future  Force  UAVs,  based  on  the 
platform’s  capabilities  and  operational  requirements  (“Four  FCS,”  2005).  The  sizes  and 
capabilities  of  the  vehicles  generally  increase  in  subsequent  classes.  Platoons  will 
operate  Class  I  airframes.  Companies  and  battalions  will  operate  the  Class  II  and  Class 
III  aircraft,  respectively.  Finally,  brigades  will  operate  the  largest  platform  consisting  of 
the  Class  IV  UAV.  The  unmanned  aircraft  will  carry  out  a  variety  of  missions  from 
communication  relays  to  target  acquisition  capabilities.  Table  4  provides  the 
performance  capabilities  of  the  FCS  UAVs. 


Class  I  UAV 

Class  II  UAV 

Class  III  UAV 

Class  IV  UAV 

Unit 

Platoon 

Company 

Battalion 

Brigade 

Weight 

5-10  lb 

100-150  lb 

300-500  lb 

>3,000  lb 

Endurance 

50  min 

2  hr 

6  hr 

6  hr 

Radius 

8  km 

16  km 

40  km 

75  km 

Transport 

manpackable 
(35  lb  system) 

2  soldier 
remount 

2  man  lift 

100  m  x  50  m 
recovery  area 

Aircraft 

Raven 

(interim) 

TBD 

Shadow 

(interim) 

Fire  Scout 

Table  4.  FCS  UAVs  Characteristics  and  Performance  Data  (After  “Unmanned 

Aircraft,”  2005). 


14 


1. 


Class  I  UAV 


i 

t 


Figure  4.  Honeywell’s  Miniature  Air  Vehicle  (From  “Ducted  Fan,”  2005). 


The  Raven  described  in  Section  A  provides  the  Interim  Force  with  a  Class  I  UAV 
capability  at  the  battalion  level.  However,  the  Army  plans  to  provide  this  capability 
down  to  platoon  level.  Honeywell  Aerospace  is  moving  towards  this  goal  by  developing 
the  backpack-size  Miniature  Air  Vehicle  (MAV)  displayed  in  Figure  4  (“Ducted  Fan,” 
2005).  In  February  2006,  Honeywell  placed  an  order  with  AAI  Corporation  to  produce 
55  MAVs  through  November  2006.  The  platforms  will  be  used  in  the  Advanced  Concept 
Technology  Demonstration  (ACTD)  program  of  the  Defense  Advanced  Research 
Projects  Agency  (DARPA)  (“AAI,”  2006).  During  the  development  phase,  AAI  will 
incorporate  several  new  design  features  into  the  UAV. 

The  Class  I  vehicles  have  been  coined  with  the  term  “hover  and  stare”  due  to  their 
ability  to  take  off  and  land  like  a  helicopter  and  provide  day  or  night  video  surveillance  as 
well  as  still  imagery.  The  design  uses  a  ducted-fan  technology  that  draws  air  in  from  the 
top  and  pushes  it  out  from  the  bottom.  The  system  is  capable  of  flying  along  at  50  mph  at 
an  operational  altitude  of  100  to  500  feet  above  the  ground  and  a  maximum  altitude  of 
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over  10,000  feet.  Current  versions  use  gasoline  and  other  heavy  fuels  common  to  U.S. 
Military  ground  vehicles  and  aircraft  (“AAI,”  2006). 

Honeywell  uses  a  flight  management  subsystem  and  micro-electrical  mechanical 
systems  (MEMS)  to  keep  the  device  lightweight  and  provide  autonomous  flight  features 
(“One  Small,”  2005).  The  13-inch  diameter,  17-pound  modular  system  provides  small 
units  with  reconnaissance  capabilities  that  soldiers  are  able  to  carry  in  an  infantryman’s 
backpack  (“AAI,”  2006). 

Another  key  reconnaissance  feature  of  the  platform  is  its  "perch  and  stare" 
capability,  when  the  MAV  acts  as  an  unattended  ground  sensor.  While  in  this  mode,  the 
vehicle  can  provide  video  surveillance  and  conserve  fuel  at  the  same  time.  When  the 
situation  changes,  the  MAV  can  lift  off  and  fly  to  another  mission  area  (Crane,  2005). 


2.  Class  II  UAV 


Figure  5.  Class  II  -  OAV  II  (From  “Organic  Air,”  2005). 


In  July  2005,  the  Army’s  Future  Combat  Systems  program  entered  into  the  first 
phase  of  development  for  the  Class  II  UAV  system.  Serving  as  the  Lead  Systems 
Integrator  (LSI)  for  the  U.S.  Army’s  FCS  program,  Boeing  (along  with  Science 
Applications  International  Corporation  (SAIC))  initiated  a  contract  with  Piasecki  Aircraft 
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Corporation  to  develop  its  version  of  the  Class  II  system  called  the  Air  Scout.  Phase  1  is 
nearing  its  planned  ten-month  schedule,  which  has  included  an  assessment  of 
requirements  and  risk  assessments  of  concepts  for  this  class  (“Four  FCS,”  2005).  Up 
until  this  point,  the  FCS  LSI  and  DARPA  have  been  developing  different  technologies 
with  the  intent  of  narrowing  down  a  final  solution  for  the  Class  II  UAV  system.  DARPA 
has  focused  on  ducted  fan  technology  resulting  in  the  Organic  Air  Vehicle  II  (OAV  II) 
displayed  in  Figure  5.  Piasecki’s  platform,  the  Air  Scout,  uses  a  non-ducted  fan  system 
and  resembles  a  smaller  version  of  the  PA-59H  Airgeep  II  (“Four  FCS,”  2005). 

By  middle  of  2006,  selected  LSI  and  DARPA  platforms  will  be  assessed  for  their 
ability  to  meet  FCS  capability  requirements.  This  assessment  will  consist  of  a  24-month 
concept  maturation  phase.  The  phase  will  conclude  with  a  flight  assessment  in  2008.  At 
this  point,  the  Class  II  UAV  will  enter  a  final  System  Design  and  Development  (SDD) 
phase.  The  Army,  LSI,  and  DARPA  will  use  a  cost-value  comparison  to  select  the  FCS 
integrated  UAV  system.  The  first  FCS  system-of-systems  testing  will  occur  in  2010  with 
a  projected  follow-on  fielding  in  2014  (“Four  FCS,”  2005). 

The  Class  II  UAV  is  a  unique  system  in  that  it  will  be  vehicle  mounted.  This  will 
enhance  the  platform’s  capability  by  allowing  it  to  take  off  and  land  in  unimproved  areas 
as  well  as  keeping  its  launch  and  recovery  assets  more  mobile.  Mobility  will  be  critical 
since  the  UAV  will  provide  reconnaissance,  surveillance,  and  target  acquisition  for 
beyond-line-of-sight  and  non-line-of-sight  engagements  at  the  company  level.  The 
system  will  also  boast  autonomous  control  and  man-in-the-loop  cueing  actions,  freeing 
up  soldiers  to  assess  the  enhanced  imagery  provided  by  the  UAV  (“Four  FCS,”  2005). 
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3. 


Class  III  UAV 


Figure  6.  Class  III  -  Prospector  (From  Latimer,  2005). 


In  October  2005,  Teledyne  Brown  Engineering  Inc.,  AAI  Corp.,  and  Piasecki 
Aircraft  Corp.  were  awarded  contracts  for  development  on  the  Class  III  UAV  system. 
The  systems  developed  by  these  companies  include  the  Prospector  (Figure  6),  Shadow  III 
and  the  Air  Guard,  respectively  (“Four  FCS,”  2005).  The  phases  of  development  for  this 
FCS  platform  are  similar  to  the  Class  II  UAV  roadmap.  Carried  out  in  three  phases,  the 
process  will  involve  the  assessment  of  different  technologies  mounted  on  a  variety  of 
platforms.  While  DARPA  chose  a  ducted  fan  technology  for  the  OAV  II,  it  selected  a 
rotorcraft  technology  for  its  Class  III  version.  LSI  is  investing  in  a  gyrocopter  and 
alternate  fixed  wing  design  for  the  system.  A  final  candidate  will  be  selected  and  ready 
for  testing  and  fielding  in  2010  and  2014  respectively  (“Four  FCS,”  2005). 

While  some  of  the  designs  for  the  Class  III  resemble  the  Class  II  platforms,  they 
are  larger  and  have  greater  payload  capacities.  These  UAVs  will  have  extended 
endurance  and  be  capable  of  carrying  sensor  packages  suited  for  reconnaissance, 
communications  relay,  early  warning,  target  acquisition  and  designation,  and  minefield 
detection  to  support  battalion  level  operations  (“Four  FCS,”  2005). 
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4. 


Class  IV  UAV 


Figure  7.  Fire  Scout  RQ-8A  (From  “Fire  Scout,”  2004). 


The  Army  selected  the  Fire  Scout  RQ-8B  as  its  Class  IV  UAV  in  2003.  Northrop 
Grumman  plans  to  deliver  two  prototypes  in  2006  as  part  of  an  eight-year  contract 
beginning  in  2003  (“Unmanned  Aircraft,”  2005;  “Fire  Scout,”  2004).  Selection  for  the 
Class  IV  platform  came  more  quickly  than  the  other  FCS  UAVs  due  to  the  Fire  Scout’s 
successful  developmental  testing  as  the  Fire  Scout  (RQ-8A)  Vertical  Take-Off  and 
Landing  (VTOL)  Tactical  UAV  (VTUAV)  program  in  the  U.S.  Navy  (“Unmanned 
Aircraft,”  2005).  By  2005,  the  Fire  Scout  (Figure  7)  had  completed  over  100  successful 
test  flights  incorporating  Tactical  Common  Data  Link  (TCDL)  operations,  multiple 
sensor  packages,  and  ground  control  systems  (“Unmanned  Aircraft,”  2005). 

Enhancements  to  the  RQ-8B  model  include  a  four-blade  rotor  system  versus  the 
three-blade  design  on  the  RQ-8A,  eight  hours  of  operational  time,  and  an  increased 
payload  weight  of  600  pounds.  The  system  also  has  an  operational  radius  of  150  nautical 
miles  (“Unmanned  Aircraft,”  2005;  “Fire  Scout,”  2004).  The  four-blade  rotor  system 
allows  the  Fire  Scout  to  carry  several  payloads  at  one  time,  increasing  its  operational 
capabilities  at  brigade  level.  This  will  allow  brigade  commanders  to  take  advantage  of 
the  General  Atomics  Lynx  Synthetic  Aperture  Radar  with  ground  Moving  Target 
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Indicator  (SAR/MTI),  an  Electro-Optical/Infrared/Laser  Designator  range  finder 
(EO/IR/LD),  and  a  communication  relay  package  simultaneously  (“Fire  Scout,”  2004). 

The  Fire  Scout  will  be  able  to  support  precision  strike  missions.  The  development 
program  has  resulted  in  the  installation  and  testing  of  two  four-packs  of  2.75-inch  rocket 
launchers  designed  to  carry  Advanced  Precision  Kill  Weapon  System  laser-guided 
rockets  and  another  laser-guided  precision  munition  called  Viper  Strike  (“Fire  Scout,” 
2004).  Table  5  provides  the  performance  capabilities  of  the  RQ-8B  Fire  Scout. 


RQ-8B 

RQ-8B 

Length 

22.9  ft 

Wing  Span 

27.5  ft 

Gross  Weight 

3,1501b 

Payload  Capacity 

600  lb 

Fuel  Capacity 

1,288  1b 

Fuel  Type 

JP-5/JP-8 

Engine  Make 

Rolls  Royce  250- 
C20W 

Power 

420  shp 

Data  Link(s) 

LOS  C2 

LOS  Video 

Frequency 

Ku-band 

UHF 

C-band 

Endurance 

6+  hr 

Max/Loiter  Speed 

125/0  kt 

Ceiling 

20,000  ft 

Operating  Radius 

150  nm 

Launch 

vertical 

Landing 

hover 

Sensor 

EO/IR/LDRF 

Sensor  Make 

FSI  Brite 
Star  II 

Table  5.  Fire  Scout  RQ-8B  Characteristics  and  Performance  Data  (After 

“Unmanned  Aircraft,”  2005). 


E.  GROUND  CONTROL  AND  MISSION  PAYLOADS 

Several  controlling  stations  and  devices  make  it  feasible  to  launch,  guide,  and 
recover  the  unmanned  vehicles.  They  also  allow  for  the  collection  and  evaluation  of 
information  provided  by  the  various  mission  payloads  on  the  platforms.  This  section 
provides  a  brief  overview  of  the  various  ground  control  systems  and  mission  payloads 
available  for  the  UAVs  described  in  this  research. 
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1.  Launch  and  Recovery  Site  (LRS) 

This  thesis  frequently  refers  to  Launch  and  Recovery  Sites  versus  a  similar  term, 
Launch  and  Recovery  Stations.  The  former  describes  the  geographic  location  where 
UAVs  leave  for  and  return  from  mission  assignments.  The  latter  refers  to  a  mobile 
hardware  component  that  assists  flight  control  during  the  launch  and  recovery  phases  of 
the  mission  as  the  UAV  takes-off  and  lands  on  the  runway.  From  here  on,  we  refer  to 
Launch  and  Recovery  Sites  as  LRSs  unless  otherwise  stated. 

ASC-U  models  the  location  of  each  LRS  in  the  scenario  with  a  grid  coordinate 
system.  During  the  length  of  the  scenario,  the  location  of  an  LRS  may  change  depending 
on  the  mission  requirement  of  its  owning  unit.  LRSs  vary  in  the  type  and  number  of 
UAVs  that  they  can  support.  Once  launched,  flight  control  for  the  unmanned  platform 
passes  to  a  Ground  Control  Station  (GCS). 

2.  Ground  Control  Station  (GCS) 

Ground  Control  Stations  house  the  electronic  hardware  to  effectively  command 
and  control  UAVs  throughout  the  mission.  The  station  usually  consists  of  a  rectangular 
shelter  mounted  on  a  vehicle  such  as  the  HMMWV  displayed  in  Figure  8.  The  shelter 
contains  pilot  stations  and  monitors  (Figure  9)  in  order  to  display  video,  still  imagery,  and 
sensor  data  provided  by  the  mission  payloads  on  the  UAV. 
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Figure  8.  Ground  Control  Station  (From  Farmer,  2001). 


While  ASC-U  is  capable  of  modeling  a  wide  range  of  GCS  configurations,  the 
simulation  currently  accounts  for  two  types:  the  One  System  Ground  Control  Station 
(OSGCS),  and  the  Dismounted  Controller  Device  (DCD).  The  GCSs  are  unique  in  their 
capability  to  control  different  types  of  UAVs,  and  have  a  maximum  capacity  of  platforms 
that  they  are  able  to  handle  at  any  given  time. 

AAI  Corporation  has  produced  the  OSGCS  for  the  Army’s  Shadow  TUAV 
program  since  2001.  The  system  has  logged  over  50,000  hours  during  training  and  real- 
world  operations.  The  One  System  GCS  has  controlled  the  Army  Hunter  platform,  the 
Marine  Corps’  Pioneer  TUAV  system,  and  the  Warrior  UAV  during  demonstration 
flights  in  2005.  In  October  2005,  AAI  Corp.  was  awarded  a  contract  to  produce  the 
OSGCS  system  for  the  Warrior  UAV.  Additionally,  OSGCS  meets  NATO’s 
standardization  agreement  (STANAG  4586),  making  it  the  control  station  of  choice 
among  allied  military  forces  (“Army  Orders,”  2005). 
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Figure  9.  Ground  Control  Station  Interior  (From  Farmer,  2001). 

The  Future  Combat  Systems  LSI  is  investigating  a  multi-asset  controller  called 
the  Dismounted  Controller  Device  (DCD).  The  soldier  carries  and  operates  the  device 
while  on  dismounted  patrols  or  riding  in  vehicles.  A  key  feature  that  currently  separates 
the  DCD  from  other  control  systems  is  its  projected  integration  with  the  FCS  family  of 
systems.  Along  with  UAVs,  requirements  call  for  the  device  to  control  several  systems 
to  include,  but  not  limited  to:  Unmanned  Ground  Vehicles  (UGVs),  Manned  Ground 
Vehicles  (MGVs),  Non-Line-of-Sight  Launch  Systems  (NLOS-LS),  Intelligent  Munitions 
(IMS),  and  Unattended  Ground  Sensors  (UGSs).  The  DCD  will  go  through  many  interim 
fieldings  as  new  technologies  are  developed.  Human  factors  engineering  will  be  essential 
to  ensure  that  the  device  does  not  interfere  with  the  soldiers’  other  mission-critical  tasks 
(“FCS  Dismounted,”  2005). 

3.  Remote  Video  Terminal  (RVT) 

The  Remote  Video  Terminal  (RVT)  is  a  scaled  down  version  of  a  GCS 
specifically  designed  to  receive  telemetry  and  video  data  directly  from  the  UAV.  The 
Army  plans  to  field  the  terminals  to  command  posts  within  battalions.  Intelligence 
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analysts  and  fire  support  officers  within  Tactical  Operation  Centers  (TOC)  can  download 
captured  images  with  telemetry  data  for  further  analysis  and  execution  of  calls  for  fire. 

A  variant  of  the  OSGCS  called  the  One  System  Remote  Video  Terminal 
(OSRVT)  is  a  mobile,  manpackable  terminal  planned  for  the  FCS  family  of  systems. 
This  device  will  feature  an  overlay  containing  icons  for  identified  enemy  units  and 
vehicles  in  order  to  enhance  target  acquisition  and  identification  (“Army  Orders,”  2005). 


4.  Mission  Payloads 

Today’s  UAVs  carry  a  variety  of  payloads  that  enhance  their  mission  capabilities. 
ASC-U  simulates  UAVs  with  several  different  payloads  depending  on  their  type  and 
intended  missions.  While  generating  the  solution,  ASC-U  matches  mission  areas 
requiring  service  by  a  payload  or  set  of  payloads  with  available  UAVs  capable  of 
carrying  the  required  payload.  The  mission  payloads  include: 

•  Electro-Optical/Infrared  (EO/IR) 

•  Laser  Rangefinder  (LR) 

•  Laser  Designator  (LD) 

•  Synthetic  Aperture  Radar/Moving  Target  Indicator  (SAR/MTI) 

•  Foliage  Penetrating  (FOPEN)  Radar 

•  Light  Detection  and  Ranging  (LIDAR)  Sensor  Technology 

•  Electronic  Warfare  (EW) 

•  Global  Positioning  System  (GPS)  Designator 

•  Chemical,  Biological,  Radiological,  Nuclear,  Explosive  (CBRNE) 

•  Weapon  Systems 

•  Mine  Detection 

•  Supply  Delivery 

•  Communication  Relay 

•  Meteorological  Sensors 

•  Signals  Intelligence  (SIGINT) 

•  Communication  Intelligence  (COMINT) 

•  Electronic  Intelligence  (ELINT) 
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This  chapter  provided  an  overview  of  the  different  UAVs  considered  in  the  study. 
We  also  described  several  controlling  stations  and  devices  that  make  it  feasible  to  launch, 
guide,  and  recover  the  unmanned  vehicles.  In  the  next  chapter,  we  introduce  ASC-U’s 
approach  to  solving  the  dynamic  routing  problem  involving  UAVs. 
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III.  SIMULATION  DESCRIPTION 


A.  UAV  ALLOCATION  PROBLEM 

The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center 
(TRAC)  and  the  Modeling,  Virtual  Environments,  and  Simulations  Institute  (MOVES)  at 
the  Naval  Postgraduate  School,  Monterey,  California  developed  the  Assignment 
Scheduling  Capability  for  UAVs  (ASC-U)  simulation  tool  to  assist  in  the  analysis  of 
UAV  requirements  for  the  current  and  future  force.  Figure  10  displays  the  study  analysis 
process.  The  process  has  evolved  within  TRAC,  separated  into  working  groups 
responsible  for  components  of  this  endeavor.  Analysts  link  tactical  scenarios  with  UAV 
mission  requirements  and  performance  capabilities.  ASC-U  uses  the  data  to  generate 
several  measures  of  effectiveness  (MOE)  and  a  flight  schedule  for  each  scenario  (Ahner, 
2005b). 


Each  scenario  provides  a  set  of  mission 
requirements.  The  screening  tool  uses  UAV  data 
to  find  feasible  UAV  mixes  that  satisfy  mission 
requirements  while  meeting  other  goals. 


UAV  Mission 
Requirements 


Operating  Environment  UAV  Missions 

METT-TC  Factors  •  Reconnaissance 

•  Target  Acquisition 

•  Communications 

•  Sensor  Payloads 


Figure  10.  Study  Analysis  Process  Diagram  (After  Ahner  et  al.,  2006). 


ASC-U  is  a  scheduling  model  that  applies  UAV  resources  against  the  schedule  of 
mission  requirements,  given  the  battlefield  conditions  within  a  scenario.  ASC-U  employs 
a  discrete  event  simulation  coupled  with  the  optimization  of  a  linear  objective  function. 
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The  model  is  based  on  the  framework  developed  by  Havens  (2002)  for  Dynamic 
Allocation  of  Fires  and  Sensors  (DAFS)  used  to  evaluate  factors  associated  with 
networking  assets  in  the  Future  Combat  Systems.  Given  a  scenario  and  a  mix  of  UAVs, 
ASC-U  determines  a  feasible  schedule  for  UAV  missions.  The  simulation  tool  considers 
airframe  capabilities,  payloads,  the  locations  and  capacities  of  ground  control  station  and 
launch  and  recovery  sites,  remote  viewing  terminals  requirements,  and  communication 
footprints  and  capacities  (Ahner  et  al.,  2006). 

The  scenario  consists  of  a  set  of  mission  area  locations  and  requirements,  friendly 
unit  locations,  platform  and  payload  characteristics,  and  ground  control  characteristics. 
Each  mission  area  has  a  set  of  sensor,  weapon,  or  communication  requirements.  The 
mission  areas  do  not  move  during  the  simulation.  Rather,  each  area  has  a  start  and  end 
time  associated  with  it.  As  the  simulation  proceeds,  mission  areas  “open”  and  “close” 
creating  a  time  window  in  which  UAVs  with  correct  capabilities  can  service  the  mission. 
The  number  and  type  of  UAVs  constrain  the  solution,  as  do  UAV  performance 
capabilities  such  as  air  speed,  operating  time,  transition  time,  and  operating  radius.  The 
number  and  capacities  of  ground  control  stations  (GCSs)  and  launch  and  recovery  sites 
(LRSs)  also  constrain  the  solution.  Additionally,  the  GCSs  and  LRSs  move  during  the 
execution  of  the  scenario  (Ahner  et  al.,  2006). 

ASC-U  employs  a  Discrete  Event  Simulation  (DES)  model  that  simulates  the 
operational  environment  of  the  UAVs.  The  simulation  is  based  on  Simkit  developed  by 
Professor  Arnold  Buss  at  the  Modeling,  Virtual  Environments,  and  Simulations  Institute 
(MOVES)  at  the  Naval  Postgraduate  School,  Monterey,  California.  Simkit  is  written  in 
Java  2™  and  is  free  to  download  at  the  Simkit  website  (Buss,  2006).  The  simulation 
component  within  ASC-U  accounts  for  the  geographical  dispersion  of  UAVs,  ground 
control  systems,  and  mission  areas  within  the  scenario.  The  DES  model  simulates  travel, 
loiter,  and  transition  times.  The  flight  schedule  takes  into  account  the  actual  routing  of 
UAVs  over  time  and  distance  and  results  in  a  realistic  and  feasible  product  for  each 
scenario. 

ASC-U  blends  optimization  and  simulation  techniques  to  solve  the  complex 

assignment  and  scheduling  problem  presented  by  the  scenarios  in  the  Army’s  UAV  Mix 
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Analysis.  It  handles  the  UAV  routing  problem  with  a  “rolling  algorithm.”  This  results  in 
an  optimal  assignment  of  each  available  UAV  to  at  most  one  mission  area  during  the 
specified  time  horizon  at  every  optimization  step.  This  problem  is  simpler  than  the 
general  UAV  scheduling  problem  because  at  any  given  point  in  time  it  restricts  the  set  of 
missions  that  will  be  considered,  as  well  as  the  number  of  missions  that  can  be  assigned 
to  a  specific  UAV.  While  ASC-U  is  not  guaranteed  to  provide  an  optimal  solution,  the 
“rolling  algorithm”  provides  a  systematic  method  of  obtaining  "good"  schedules  with  a 
reasonable  amount  of  computing  effort.  The  impacts  of  the  time  horizon  and  the 
optimization  interval  on  solution  quality  are  evaluated  as  part  of  this  study. 

B.  DYNAMIC  PROGRAMMING  APPROACH 

ASC-U  uses  an  approximate  dynamic  programming  approach  known  as  a  “rolling 
horizon”  approach.  This  method  accounts  for  state  transitions  within  the  simulation 
rather  than  modeling  them  in  an  optimization  formulation.  The  approach  is  able  to  spiral 
in  new  problem  characteristics  such  as  UAV  attrition  and  enable  many  measures  of 
effectiveness  to  be  collected  and  reported  (Ahner,  2005b). 

ASC-U  obtains  an  optimal  solution  to  a  simplified  problem  by 
assigning  each  available  UAV  to  at  most  one  mission  that  is  currently  available  or  will  be 
available  within  a  future  time  horizon.  The  time  horizon  is  unique  to  each  type  of  UAV. 
The  simulation  derives  a  mission  value  for  each  assignment  by  considering  the  amount  of 
time  each  UAV  is  on-station  at  a  mission  area  and  a  value  rate  for  sensor  requirement 
within  a  mission  area.  The  time  horizon  is  used  to  identify  the  missions  that  become 
available  and  does  not  influence  the  value  of  the  service  time.  The  simulation  schedules 
UAVs  to  leave  LRSs  and  arrive  at  mission  areas  in  time  to  achieve  the  maximum  value 
for  servicing  the  area.  The  process  advances  to  the  next  optimization  interval  and 
assesses  all  UAVs  not  yet  launched  even  if  they  were  scheduled  to  launch  in  the  previous 
optimization  period.  Available  UAVs  are  assigned  to  missions  that  do  not  yet  have  a 
UAV  scheduled  to  that  mission  (Ahner,  2005a). 

Figure  11  displays  the  dynamic  process  of  assigning  two  UAVs  to  five  mission 
areas.  The  optimization  interval  for  the  scenario  is  set  at  t  simulation  hours.  The  arrows 
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at  point  1  indicate  the  first  (to)  and  second  (0)  optimization  periods.  At  time  to,  two 
UAVs  are  available,  UAV  1  and  UAV  2.  ASC-U  assesses  available  UAVs  with  GCS 
resources  and  available  missions  within  the  time  horizon  spanning  to  to  h.  Within  this 
time  horizon,  missions  1,  2,  and  3  are  considered.  At  point  2,  the  UAVs  with  appropriate 
sensor  payloads  are  assigned  to  mission  areas  requiring  service  by  such  a  payload.  For 
our  example,  assume  that  it  is  best  to  assign  UAV  1  and  UAV  2  to  missions  1  and  2, 
respectively.  The  simulation  schedules  UAV  1  to  launch  immediately,  accounts  for  its 
flight  time,  and  schedules  its  arrival  at  mission  1  as  indicated  by  point  3.  Notice  that 
UAV  2  is  scheduled  to  launch  shortly  after  t\  in  order  to  arrive  just  in  time  to  begin 
service  at  mission  2.  At  each  optimization  interval,  ASC-U  considers  all  available 
UAVs,  this  includes  aircraft  that  are  scheduled  but  not  launched.  At  time  t\,  UAV  2  is 
scheduled  but  not  launched,  and  is  therefore  “unassigned”  and  considered  as  an  available 
UAV  for  the  second  optimization  period  as  indicated  at  point  4.  Now,  the  time  horizon 
includes  missions  1,  2,  3,  and  4.  UAV  2  achieves  a  higher  assignment  value  for  serving 
mission  4  then  it  did  for  mission  2  and  reschedules  to  launch  at  point  5.  The  simulation 
does  not  reassign  UAVs  while  in  flight.  In  this  manner,  the  simulation  builds  a  flight 
schedule  based  on  the  optimal  assignments  achieved  within  the  fixed  time  horizons 
(Ahner  et  al.,  2006). 


Figure  1 1 .  Assignment  Dynamics  Example  (After  Ahner  et  al.,  2006). 
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The  simulation  determines  the  mission  value  for  flying  a  UAV  to  each  mission 
using  a  valuation  rate  for  sensor  requirements  on  each  mission  area.  The  sensor  valuation 
rates  depend  on  user-input.  This  gives  ASC-U  the  flexibility  of  use  as  an  analytical  tool 
for  UAV  mix  studies  as  well  as  by  operational  commands  that  wish  to  prescribe  priorities 
to  different  missions.  The  UAV’s  time-on-station  for  available  missions  is  multiplied  by 
the  value  rate  in  order  to  calculate  the  mission  value  for  each  UAV-mission  pairing 
(Ahner  et  al.,  2006). 

Figure  12  displays  the  dynamic  cueing  and  transition  process  conducted  by  ASC- 
U  in  order  to  build  the  UAV  flight  schedule.  The  Dynamic  Simulation  State  Transition 
indicates  the  state  of  the  system  immediately  prior  to  the  next  optimization  interval.  For 
discussion,  let  the  simulation  state  exist  at  time  zero.  Airframes  are  located  with  their 
respective  LRSs  and  dispersed  according  to  the  scenario’s  initial  battlespace 
configuration.  The  simulation  triggers  its  first  optimization  interval  as  we  saw  in  Figure 
11.  The  optimization  sequence  employs  the  Value  of  Potential  Assignment  Generator  to 
calculate  the  value  for  all  possible  UAV-mission  pairings  that  occur  within  the  time 
horizons  of  each  UAV  by  type.  The  Generator  considers  movement,  ranges,  and 
capacities  of  the  ground  control  systems  and  passes  the  data  along  with  the  mission 
values  to  the  Optimization.  The  Optimization  makes  assignments  based  on  the  maximum 
value  obtained  for  each  UAV-mission  pairing  and  sends  the  results  to  the  UAV 
Scheduler.  The  UAV  Scheduler  determines  launch  times  for  assigned  UAVs  resulting  in 
a  flight  schedule  that  accounts  for  travel  and  service  times  for  each  mission  area.  The 
simulation  proceeds  to  execute  all  ingress  and  egress  flights  as  well  as  pay  load/ sensor 
actions  on  assigned  mission  areas  that  occur  before  the  next  optimization  interval.  This 
continues  until  the  completion  of  the  scenario  (Ahner  et  al.,  2006). 
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Figure  12.  ASC-U  Dynamic  Cueing  and  Transitions  (After  Ahner  et  al.,  2006). 


This  thesis  investigated  a  scenario  involving  21,360  mission  areas.  The 
parameters  for  the  mission  areas  are  represented  in  a  Microsoft®  Excel®  spreadsheet  and 
loaded  into  ASC-U  prior  to  each  simulation  run.  Currently,  mission  areas  implemented 
in  ASC-U  capture  the  following  details:  Cartesian  coordinates  for  location;  start  and  end 
times;  mission  package  requirements;  value  rate  for  allocating  UAV  assets;  and  Army 
Universal  Task  List  (AUTL)  tasks  associated  with  each  mission.  Figure  13  displays  a 
Cartesian  coordinate  plot  of  the  mission  areas  and  ground  control  systems.  We  note  that 
TRAC  developed  several  larger  scenarios  (which  consist  of  over  40,000  mission  areas) 
during  and  after  the  data  analysis  phase  of  this  thesis.  We  were  not  able  to  explore  the 
additional  scenarios  within  this  thesis  due  to  the  unavailability  of  data. 
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Figure  13.  Cartesian  Coordinate  Plot  of  Mission  Areas  and  Ground  Control  Systems. 

C.  CONSTRAINTS,  LIMITATIONS,  AND  ASSUMPTIONS 

This  section  describes  the  constraints,  limitations,  and  assumptions  made  by  the 
developers  of  ASC-U.  Constraints  represent  uncontrollable  conditions  that  influenced 
the  development  of  the  simulation.  Limitations  represent  conditions  that  the  developers 
defined  to  scope  the  solution.  Assumptions  represent  choices  made  by  the  developers  to 
simplify  the  problem  into  a  manageable  set  of  components  in  order  to  generate  a  solution 
(Ahner  et  al.,  2006). 

1.  Constraints 

Two  conditions  acted  as  constraints  during  the  development  of  ASC-U,  problem 
type  and  time.  The  problem  type  is  a  combination  of  vehicle  assignment  and  routing 
problems.  Similar  problems  have  been  identified  as  “NP-hard”  and  hinder  analysts’ 
ability  to  generate  a  solution  in  a  polynomial  number  of  computational  steps  (Ahner  et 
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al.,  2006).  The  difficulty  lies  in  the  process  of  enumerating  all  of  the  UAV-mission 
assignments  in  order  to  develop  the  optimal  schedule  within  a  reasonable  amount  of  time. 

The  developers  were  also  limited  by  time  available  to  construct  a  tool  required  for 
the  Army’s  UAV  Mix  Analysis.  Once  developed,  however,  ASC-U  leveraged  the  ability 
to  incorporate  additional  capabilities  and  enhancements.  The  simulation  is  able  to  spiral 
in  new  problem  characteristics  and  maintain  flexibility  in  future  studies  (Ahner  et  al., 
2006). 


2.  Limitations 

The  following  is  a  list  of  limitations  from  the  ASC-U  Users/Analyst  Manual 
(Ahner  et  al.,  2006): 

•  ASC-U  represents  mission  areas  as  a  single  Cartesian  coordinate 
for  the  entire  mission  window.  If  more  than  one  UAV  is  required 
to  perform  a  mission,  the  user  must  provide  sensor  requirements  at 
two  different/adjacent  coordinates.  The  use  of  single  coordinates 
for  mission  areas  account  for  the  first  order  effects  of  assignments 
over  space  and  time  and  GCS  capacity  and  range  constraints. 

•  ASC-U  does  not  account  for  tactics,  techniques,  and  procedures 
concerning  the  employment  of  UAVs  as  they  serve  mission  areas. 

•  The  UAV  remains  at  the  mission  area  until  all  sensor  requirements 
that  the  UAV  can  perform  end  or  the  UAV  must  return  to  its  LRS 
so  that  it  does  not  exceed  its  operating  time. 

•  ASC-U  does  not  model  passing  UAV  control  between  two  GCSs. 
A  single  GCS  controls  a  UAV  throughout  its  entire  mission. 

•  The  user  defines  the  contents  of  a  sensor  package  for  each  UAV 
type  prior  to  running  the  simulation.  Sensor  packages  cannot  be 
reconfigured  during  the  simulation. 

•  The  user  provides  LRS  and  GCS  locations  for  particular  times 
throughout  the  length  of  the  scenario.  LRSs  and  GCSs  travel  at  a 
uniform  rate  between  the  locations  during  the  prescribed  times. 
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3.  Assumptions 

The  following  is  a  list  of  assumptions  from  the  ASC-U  Users/ Analyst  Manual. 
The  authors  note  that  some  limitations  may  be  considered  as  assumptions  in  order  to 
highlight  their  effect  on  the  solution  (Ahner  et  ah,  2006). 

•  The  overall  mission  value  for  the  scenario  is  the  sum  of  the  sensor 
requirement  values  for  served  mission  areas  making  up  the 
solution. 

•  UAV  airspeed  and  operating  radius  are  constant. 

•  UAV  control  and  data  link  requirements  are  satisfied. 

•  Transition  times  between  flights  are  constant. 

•  Transition  times  represent  recovery  of  UAVs  and  include 
swapping  payloads. 

•  The  simulation  does  not  consider  attrition  of  or  damage  to  UAVs. 
Additionally,  maintenance  requirements  are  not  considered. 

•  Army  Airspace  Command  and  Control  (A2C2)  requirements  are 
satisfied. 

•  The  locations  of  the  GCSs  and  LRSs  locations  are  extracted  from  a 
Map  Exercise  (MAPEX)  or  a  simulation  running  an  approved 
scenario  and  are  known  throughout  the  scenario. 

•  LRSs  and  GCSs  travel  at  a  uniform  rate  between  locations  from 
start  to  end  times. 

•  GCSs  often  operate  in  pairs,  employing  a  “leap  frog”  movement 
where  one  controls  UAVs  from  a  forward  position  while  the  other 
GCS  moves  to  its  next  position.  ASC-U  replicates  this  method  of 
employment  by  representing  two  GCSs  by  a  single  GCS  that 
operates  continuously  with  uniform  movement  between  positions. 
This  modeling  technique  allows  for  uninterrupted  GCS  capacity 
and  control  between  positions. 

•  LRSs  are  within  range  of  at  least  one  GCS  for  launch  and  recovery 
operations. 

•  Terrain  features  do  not  influence  UAV  mission  accomplishment. 
ASC-U  does  not  model  terrain  features  within  the  simulation. 

•  Atmospheric  conditions  do  not  influence  UAV  mission 
accomplishment.  ASC-U  does  not  model  atmospheric  conditions 
within  the  simulation. 


35 


D.  ASC-U  INPUT 


The  mission  requirements  and  UAV  performance  characteristics  are  represented 
in  a  Microsoft®  Excel®  spreadsheet.  The  user/analyst  loads  the  spreadsheet  into  ASC-U 
prior  to  each  simulation  run.  The  Excel®  file  has  several  worksheets,  each  provide  data 
for  a  specific  component  of  the  mission  requirements.  Table  6  displays  the  name  of  the 
worksheets  included  in  the  Excel®  input  file. 

Our  research  explored  the  effects  of  twenty-six  simulation  and  UAV  performance 
factors  on  the  mission  value  derived  when  allocating  UAVs  to  mission  areas.  The  factors 
consisted  of  the  following:  optimization  interval,  time  horizons,  air  speed,  operating 
time,  operating  radius,  and  transition  time.  While  the  optimization  interval  is  a  single 
value  for  the  entire  scenario,  the  other  five  factors  are  unique  for  each  type  of  UAV.  The 
MetaData  worksheet  stores  the  optimization  interval  and  the  AirFrame  worksheet  lists  the 
UAV  performance  factors.  The  following  sections  provide  a  description  of  the  worksheet 
components  used  for  the  research  conducted  in  this  thesis.  For  a  complete  description  of 
the  input  worksheets  used  by  ASC-U,  see  the  current  version  of  the  ASC-U 
Users/ Analyst  Manual  (Ahner  et  ah,  2006). 


Worksheet  Title 

MetaData _ 

Mission _ 

AirFrame _ 

MissionPackageTypes 

MissionPackageLocation 

GCSInput _ 

LRSInput _ 

RVTInput _ 

SuperMissionData _ 


Table  6.  ASC-U  Input  Worksheets  (From  Ahner  et  al.,  2006). 
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1.  Meta  Data  Worksheet 

The  MetaData  worksheet  contains  three  elements:  scenario  length,  optimization 
interval,  and  report  interval  (Ahner  et  ah,  2006).  We  investigated  the  optimization 
interval  in  this  thesis. 

•  Scenario  length  indicates  the  length  of  the  scenario  in  simulation 
hours.  (Defaults  to  360.0,  if  MetaData  worksheet  is  not  present.) 

•  Optimization  interval  is  the  time  (simulation  hours)  between 
optimizations  that  derive  the  UAV  allocations  at  that  moment  in 
time.  (Optional:  Defaults  to  1.0) 

•  Report  interval  indicates  the  phases  of  the  scenario  for  which 
measures  of  performance  will  be  reported.  Measures  of 
performance  for  the  overall  scenario  length  are  also  provided  in 
addition  to  these  interval  reports.  (Optional:  Defaults  to  4.0) 


2.  Air  Frame  Worksheet 

The  AirFrame  Worksheet  contains  all  performance  characteristics  for  each  UAV. 
Each  row  in  the  worksheet  represents  one  UAV.  We  varied  the  inputs  at  each  row  under 
the  following  columns:  transition  time,  max  speed  (air  speed),  operating  radius, 
operating  time,  and  time  horizon.  The  following  list  describes  the  column  headings  for 
the  AirFrame  worksheet  (Ahner  et  ah,  2006): 

•  Type  is  the  class  of  UAV  and  may  be  used  to  ensure  that  a  proper 
LRS  is  used  for  take  off  and  landing. 

•  Unit  is  the  owning  military  unit  of  the  UAV  listed  on  the  same  row 
and  under  the  column  Type. 

•  Compatible  GCS  indicates  the  type  of  GCS  from  the  GCS 
Worksheet  that  is  able  to  control  this  UAV.  (Not  used  in  the 
current  version,  but  should  not  be  deleted.) 

•  Starting  LRS  is  the  name  from  the  LRSInput  Worksheet  of  the 
LRS  from  which  the  UAV  originates. 

•  Ending  LRS  is  the  name  from  the  LRSInput  Worksheet  of  the 
LRS  to  which  the  UAV  lands.  (Should  be  the  same  as  the  Starting 
LRS.) 

•  Transition  time  is  the  time,  upon  landing,  that  is  required  before 
the  UAV  is  available  to  launch. 

•  Max  speed  is  the  travel  rate  of  the  UAV  between  locations. 
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•  Operating  radius  is  the  max  distance  the  UAV  can  be  from  a 
controlling  GCS  for  effective  control. 

•  Name  is  a  unique  name  for  the  UAY. 

•  StartAvail  is  the  time  the  UAV  is  initially  available  for  launch, 
usually  an  H-hour. 

•  EndAvail  is  the  time  the  UAV  is  last  available  in  the  scenario. 

•  Operating  time  is  the  maximum  time  the  UAV  may  be  away  from 
an  LRS. 

•  Time  horizon  is  the  time  window  used  in  the  rolling  algorithm  of 
the  optimization.  Time  horizon  is  usually  1  to  1.5  times  the 
operating  time  plus  transition  time.  This  value  is  critical  to 
determining  what  missions  will  be  considered  in  the  look-ahead 
time  window  to  launch  a  UAV  if  GCS  capacity  is  available. 


38 


E.  ASC-U  OUTPUT 

ASC-U  saves  the  simulation  results  as  output  reports  in  a  Microsoft®  Access® 
database.  The  output  reports  are  organized  into  fourteen  database  tables.  Table  7 
provides  a  description  for  each  report.  For  this  thesis,  we  required  the  data  contained  in 
the  Coverage  and  CoverageByType  tables. 


Table  Name 

Description 

Coverage 

The  amount  of  time  each  mission  is  covered  by  a  UAV 

CoverageByT  ype 

The  amount  of  time  each  mission  is  covered  by  each 
different  type  of  UAV 

CoverageDelay 

The  delay  from  the  time  a  mission  opens  until  it  is  first 
covered  by  a  UAV 

GCSLoadOverTime 

The  time  average  number  of  UAVs  using  each  GCS  in  each 
time  period 

GCSUtilization 

The  time  average  load  on  each  GCS 

MissionAssignment 

A  schedule  of  which  UAV  is  covering  which  missions 

MissionPackageUtilization 

The  time  average  utilization  rate  of  each  type  of 
MissionPackage  at  each  LRS 

Runlnformation 

Information  about  the  DAFS  version  and  data  used  to  create 
the  output  tables 

RVTCoverage 

Not  currently  used 

Schedule 

A  schedule  of  UAV  sorties 

T  erseRVTCoverage 

A  summary  of  the  coverage  of  each  mission  by  each  RVT 

UA  VReadyT  imeOverT  ime 

For  each  UAV,  the  amount  of  time  it  is  ready  for  launch  in 
each  time  period 

UAVUtilization 

UAV  utilization  data 

UAVUtilizationOverT  ime 

For  each  UAV,  the  proportion  of  ready  time  that  the  UAV 
was  airborne  in  each  time  period 

Table  7.  Measures  of  Performance  Tables  (From  Ahner  et  al.,  2006). 


This  chapter  introduced  the  UAV  allocation  problem  and  the  ASC-U  simulation 
tool.  We  limited  our  discussion  to  the  components  of  ASC-U  that  we  used  in  our 
research.  We  also  point  out  that  ASC-U  is  in  continuous  development.  During  our 
research,  the  developers  added  several  new  functionalities  involving  sensor  packages, 
UAV  characteristics,  and  mission  areas.  We  used  ASC-U  version  1.3.0  modified  on 
April  13  10:41 : 1 1  PDT  2006  for  our  study. 
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IV.  ANALYSIS  METHODOLOGY 


This  chapter  discusses  the  analysis  methodology  employed  in  our  research.  We 
describe  the  selection  of  factors  included  in  our  experiment,  the  development  of  design 
points  using  Nearly  Orthogonal  Latin  Hypercubes  (NOLH),  and  the  statistical  software 
selected  for  the  analysis. 

A.  MEASURE  OF  EFFECTIVENESS  (MOE) 

The  ASC-U  simulation  supports  the  generation  of  a  number  of  MOEs,  including, 
percent  mission  coverage,  percent  mission  type  coverage,  percent  mission  coverage  by 
UAV  type,  UAV  utilization,  mission  package  utilization,  coverage  delay,  percent  AUTL 
(Army  Universal  Task  List)  coverage,  Remote  Viewing  Station  (RVT)  coverage,  and 
Ground  Control  Station  (GCS)  utilization.  Other  MOEs  provide  more  detail  that  are 
summarized  by  these  aggregate  MOEs.  An  additional  MOE  supported  by  ASC-U  is  a 
term  within  the  simulation-optimization  process  called  mission  value.  Generated  during 
the  allocation  of  UAVs  to  mission  areas,  mission  value  is  a  quantifiable  measurement  that 
captures  a  UAVs  service  to  a  mission  area.  Since  mission  value  is  representative  of  the 
percent  mission  coverage  and  based  on  the  mission  payload,  UAV  type,  and  service  time 
of  a  mission  area,  we  felt  that  it  would  provide  an  appropriate  measure  into  the  overall 
performance  of  the  simulation  and  quality  of  the  schedule  it  produces. 


B.  DESIGN  OF  EXPERIMENT 

The  research  conducted  in  this  thesis  takes  advantage  of  experimental  design 
techniques  based  on  statistical  theory  and  developed  to  assist  researchers  in  the  analysis 
of  computer  simulations  (Kleijnen  et  al.,  2005).  The  primary  goal  of  experimental  design 
is  to  assess  how  changes  to  input  parameters  (factors)  affect  the  results  (responses).  We 
applied  these  techniques  with  runs  of  the  ASC-U  simulation  tool  in  order  to  answer  the 
research  questions  and  provide  insights  into  the  model’s  development  and  application 
across  a  robust  set  of  future  mission  scenarios. 
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The  origins  of  Design  of  Experiments  (DOE)  have  their  roots  in  agricultural 
research  in  the  early  1900s  and  have  since  captured  the  attention  of  several  industries 
(Kleijnen,  2004).  Original  DOEs  proved  useful  where  only  a  few  factors  were 
considered  in  the  investigation  of  the  process  in  question.  Recently,  this  “parallel 
experimentation”  methodology  has  found  its  way  into  the  pharmaceutical  industry  where 
process  chemists  are  looking  to  enhance  drug  design  and  development  (Henry,  2002). 
With  the  advent  of  faster  computing  power  and  increased  appreciation  for  simulation 
applications,  research  analysts  have  begun  to  expand  the  application  of  DOE  to 
simulations  analysis  (Kleijnen  et  al.,  2005;  Kelton,  2001). 


1.  Factor  Selection 

ASC-U  relies  on  several  types  of  input  factors  to  model  the  scenario  and  process 
the  allocation  of  UAVs  to  mission  areas  requiring  different  mission  payloads.  We 
selected  factors  that  fell  into  two  categories:  UAV  characteristics  and  performance 
capabilities,  and  optimization  parameters. 

The  first  set  of  factors  we  considered  included  the  following  UAV  characteristics 
and  performance  capabilities:  UAV  type,  sensor  package,  operating  time,  air  speed, 
operating  radius,  and  transition  time.  The  UAVs  considered  at  the  time  of  the  study 
included  the  Extend  Range  Multipurpose  (ERMP)  UAV,  and  the  Future  Force  UAVs 
classified  as  Class  I,  II,  III,  and  IV  platforms.  The  base  scenario  used  single  values  for 
operating  time,  air  speed,  operating  radius,  and  transition  times  based  on  current  and 
future  projected  UAV  capabilities  (“Unmanned  Aircraft,”  2005). 

The  second  set  of  factors  implemented  within  ASC-U  involves  the  optimization. 
The  factors  include  the  optimization  interval  and  the  time  horizon  used  to  assess  and 
assign  UAVs  to  available  mission  areas.  The  optimization  interval  is  set  at  a  single  value 
for  the  entire  simulation  run.  The  time  horizon  is  used  to  identify  the  missions  that 
become  available  for  each  type  of  UAV.  This  length  of  time  can  be  viewed  as  a 
“planning  window”  that  is  specific  to  each  type  of  UAV  and  is  based  on  its  operational 
capabilities.  The  simulation  considers  this  future  time  horizon  when  allocating  UAVs  to 
mission  areas  at  each  optimization  interval. 
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We  based  our  final  selection  of  the  factors  explored  in  the  study  on  their 
relationship  to  the  ASC-U  solution  and  the  research  questions.  The  fact  that  ASC-U 
treats  the  assignment  of  UAVs  as  a  dynamic  routing  problem  indicated  that  influential 
factors  would  most  likely  consist  of  the  platforms’  characteristics  and  performance 
capabilities.  This  led  to  the  selection  of  the  following  four  factors  for  each  type  of  UAV: 
operating  time,  air  speed,  operating  radius,  and  transition  time.  Additionally,  we  selected 
the  time  horizon  for  each  UAV  based  on  its  incorporation  in  the  solution  formulation  and 
its  potential  impact  on  UAV  assignments. 

We  selected  the  optimization  interval  as  an  additional  factor.  The  optimization 
interval  is  a  characteristic  of  ASC-U’s  optimization,  rather  than  that  of  the  platforms  or 
the  scenario.  As  the  simulation  proceeds,  ASC-U  queries  the  status  of  all  UAVs  and 
mission  areas  in  order  to  optimize  the  allocation  of  UAVs  to  mission  areas  within  their 
respective  time  horizons.  The  simulation  tool  uses  the  optimization  interval  to  determine 
how  often  this  query  process  occurs.  An  interval  of  10,  for  example,  conducts  an 
optimization  sequence  every  ten  simulation  hours.  Our  initial  expectation  was  that  the 
smaller  the  interval,  the  higher  the  overall  mission  value  assigned  to  the  scenario’s 
schedule. 

Our  assessment  of  appropriate  design  factors  led  to  the  selection  of  26  factors 
consisting  of  five  factors  for  each  UAV  type  and  the  simulation’s  optimization  interval 
parameter.  Table  8  displays  the  factors  along  with  their  base  scenario  values  and  the 
ranges  used  in  the  DOE.  The  ranges  for  the  UAV  factors  were  developed  by  adding  and 
subtracting  20%  of  the  base  value.  We  set  the  range  for  the  optimization  interval 
between  one  and  ten  simulation  hours. 
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UAV 

Types 

Factors 

Factor 

Name 

Base  Scenario 
Value 

DOE  Ranges 

Low 

High 

ERMP 

Time  Horizon  (hr) 

EHoriz 

36 

28.8 

43.2 

Operating  Time  (hr) 

EOprT 

36 

28.8 

43.2 

Air  Speed  (km/hr) 

ESpeed 

241 

192.8 

289.2 

Operating  Radius  (km) 

ERadius 

500 

400 

600 

Transition  Time  (hr) 

ETrans 

0.5 

0.40 

0.60 

Class  I 

Time  Horizon  (hr) 

lHoriz 

0.83 

0.664 

0.996 

Operating  Time  (hr) 

lOprT 

0.83 

0.664 

0.996 

Air  Speed  (km/hr) 

1 Speed 

93 

74.4 

111.6 

Operating  Radius  (km) 

1  Radius 

8 

6.40 

9.60 

Transition  Time  (hr) 

1  Trans 

0.16 

0.128 

0.192 

Class  II 

Time  Horizon  (hr) 

2Horiz 

2 

1.60 

2.40 

Operating  Time  (hr) 

20prT 

2 

1.60 

2.40 

Air  Speed  (km/hr) 

2Speed 

93 

74.4 

111.6 

Operating  Radius  (km) 

2Radius 

16 

12.8 

19.2 

Transition  Time  (hr) 

2Trans 

0.24 

0.192 

0.288 

Class  III 

Time  Horizon  (hr) 

3Horiz 

6 

4.80 

7.20 

Operating  Time  (hr) 

30prT 

6 

4.80 

7.20 

Air  Speed  (km/hr) 

3 Speed 

222 

177.6 

266.4 

Operating  Radius  (km) 

3  Radius 

40 

32.0 

48.0 

Transition  Time  (hr) 

3Trans 

1 

0.80 

1.20 

Class  IV 

Time  Horizon  (hr) 

4Horiz 

6 

4.80 

7.20 

Operating  Time  (hr) 

40prT 

6 

4.80 

7.20 

Air  Speed  (km/hr) 

4Speed 

231 

184.8 

277.2 

Operating  Radius  (km) 

4Radius 

75 

60.0 

90.0 

Transition  Time  (hr) 

4Trans 

0.5 

0.40 

0.60 

ASC-U 

Parameter 

Factor 

Factor 

Name 

Base  Scenario 
Value 

Low 

High 

Optimization  Interval 

Optlnt 

1 

1 

10 

Table  8.  Factors  Based  on  UAV  Performance  Capabilities  and  their  Ranges. 
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2.  Nearly  Orthogonal  Latin  Hypercube  (NOLH) 

Once  we  selected  the  input  factors,  the  next  step  was  deciding  how  to  assess  the 
impact  of  the  various  levels  for  each  factor  against  the  MOEs  generated  by  ASC-U. 
Without  knowledge  of  experimental  designs  proposed  by  Kleijnen  et  al.  (2005),  an 
analyst  may  decide  to  start  with  only  two  or  three  factors  set  at  the  low  and  high  levels. 
Using  two  levels  for  three  factors  would  create  eight  design  points.  In  order  to  gain  a 
better  understanding  of  the  response  surface,  we  would  like  to  increase  the  number  of 
factors  and  expand  the  levels  for  each. 

Consider  a  full  factorial  experiment  with  our  factors  and  three  levels  (low,  middle, 
high)  in  order  to  identify  non-linear  relationships  within  the  model.  This  would  result  in 
326  (2.54  x  1012)  design  points.  Since  ASC-U  is  a  deterministic  simulation,  only  one  run 
per  design  point  would  be  required.  Nevertheless,  at  approximately  three  hours  per  run, 
even  with  1000  computers  simultaneously  executing  a  different  design,  it  would  take 
nearly  nine-thousand  centuries  to  acquire  the  data  set.  Fortunately,  there  are  effective 
alternatives  to  using  a  full  factorial  design. 

We  selected  the  space-filling  design  of  Nearly  Orthogonal  Latin  Hypercubes 
(NOLH)  to  provide  an  exploration  of  the  outer  and  interior  regions  of  the  response 
surface.  NOLH  designs  allow  the  analyst  to  develop  a  comprehensive  set  of  explanatory 
variables  represented  in  the  model  (Cioppa,  2002).  Instead  of  being  restricted  to  two  or 
three  levels,  the  analyst  can  create  a  design  that  uses  multiple  levels  or  even  a  continuous 
range  of  values  for  each  factor.  This  facilitates  the  identification  of  non-linearities  within 
the  response  surface  (Brown,  2000). 

In  addition  to  good  space-filling  properties,  orthogonality  is  another  desirable 
property  of  designs.  Orthogonal  designs  have  no  linear  relationship  between  the 
regressors  (Montgomery,  Peck,  &  Vining,  2001).  The  NOLH  technique  minimizes  the 
correlation  between  factor  columns,  creating  a  nearly  orthogonal  design  matrix.  We  can 
examine  the  off-diagonal  elements  within  the  correlation  matrix  of  the  design  in  order  to 
measure  the  level  of  orthogonality.  The  spreadsheet  design  tool  described  below  has 
several  features  that  guard  against  multicollinearity  and  produce  a  robust  space-filling 
design. 
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Using  the  N0LHdesigns_v4.xls  spreadsheet  developed  by  Professor  Susan  M. 
Sanchez,  we  generated  257  design  points  consisting  of  26  factors  (Sanchez,  2005).  The 
spreadsheet  in  Figure  14  consists  of  worksheets  that  create  designs  for  a  specific  number 
of  factors.  Since  our  experiment  called  for  26  factors,  we  used  the  worksheet  that  creates 
a  nearly  orthogonal  design  for  22-29  factors.  The  spreadsheet  employs  the  algorithm 
described  by  Cioppa  (2002)  where  the  maximum  number  of  factors  examined  in  a  Latin 

fm-n 


hypercube  is  m  + 


and  m  is  an  integer  greater  than  1 . 


\  ^  J 

the  number  of  factors  gives  m  =  8.  The  number  of  n  design 
n  =  2m  + 1  and  results  in  257  (Cioppa,  2002). 


Solving  for  m  using  29  as 
points  required  is  given  by 


Figure  14.  Nearly  Orthogonal  Latin  Hypercube  (NOLH)  Design  Spreadsheet. 
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Our  final  design  consisted  of  a  257  x  26  matrix  with  the  largest  correlation  of 
0.0816.  Figure  15  provides  the  two-dimensional  projections  for  the  optimization  interval 
and  ERMP  factors,  which  display  the  space-filling  properties  of  the  design.  We  created 
Microsoft®  Visual  Basic®  macros  to  generate  257  scenario  templates  in  Excel®  and 
transfer  the  respective  design  points  to  each  one.  At  this  point,  we  were  ready  to  begin 
our  runs. 
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Figure  15.  Scatterplot  Matrix  for  Optimization  Interval  and  ERMP  Factors. 

3.  Computing  Resources 

Simulation  analysis  requires  scenario  productions  runs.  Simulations  involving 
stochastic  processes  require  multiple  runs  per  scenario.  For  example,  a  simulation  with 
257  design  points  may  require  50  runs  each.  This  amounts  to  12,850  runs  for  a  single 
data  set.  Researchers  often  look  to  supercomputing  clusters  to  support  their  simulation 
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requirements.  Two  organizations  that  facilitate  data  farming  in  this  way  are  the  Maui 
High-Performance  Computing  Center  (MHPCC)  at  the  University  of  Hawaii  and  the 
MITRE  Corporation  in  Woodbridge,  VA  (Wolf,  2003;  Hakola,  2004).  A  script  is  usually 
created  to  transfer  each  row  of  the  design  point  to  a  single  simulation  run.  Our  computer 
resource  needs  were  different  in  two  ways. 

First,  ASC-U  is  a  deterministic  simulation  requiring  only  one  run  per  design 
point.  This  limited  the  number  of  runs  that  we  needed  to  257.  Second,  ASC-U  uses  a 
graphic  user  interface  (GUI)  to  load  each  design  point  and  initiate  a  run.  Once  the 
simulation  is  complete,  the  analyst  saves  the  results  to  a  Microsoft®  Access®  database. 
While  we  would  have  liked  to  create  a  script  to  automate  the  run-initiation  process,  we 
found  that,  based  on  time  constraints,  it  was  easier  to  initiate  each  run  manually  for  our 
study.  At  approximately  three  hours  per  run,  we  require  over  700  hours  of  processing 
time.  We  used  the  computer  labs  in  the  Operations  Research  Department,  Naval 
Postgraduate  School,  Monterey,  CA  to  complete  our  simulation  runs.  We  employed 
approximately  60  computers  simultaneously  to  load  and  initiate  each  run.  The  computing 
resources  consisted  of  Dell  2.80  GHz  Pentium  4  Computers  with  1.00  GB  RAM  running 
Microsoft®  Windows®  XP  Professional. 

C.  ANALYSIS  OF  THE  DATA 

This  section  describes  the  data  consolidation  process  and  statistical  software 
package  used  in  the  study.  We  were  interested  in  ways  to  facilitate  our  current  as  well  as 
future  studies  involving  the  ASC-U  simulation  tool.  Efforts  were  made  to  develop  a 
DOE  methodology  that  could  be  integrated  with  ASC-U.  Additionally,  we  desired  a 
user-friendly  data  analysis  package  that  facilitated  analysis  and  provided  a  graphical 
display  of  intermediate  and  final  results. 


1.  Data  Consolidation 

ASC-U  provides  the  ability  to  write  the  results  in  several  different  database 
structures.  Since  the  results  from  trial  runs  were  too  large  for  Excel®  to  handle,  we  saved 

the  outputs  to  Microsoft®  Access®  databases.  We  generated  257  databases  after 
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completing  the  runs.  Each  database  was  approximately  30  megabytes  in  size,  with  14 
tables  comprised  of  MOE  data  and  a  UAV  flight  schedule.  Our  task  was  to  extract  the 
desired  MOEs  and  consolidate  the  results  with  the  original  design  matrix. 

We  determined  that  it  was  beneficial  to  extract  the  percent  mission  type  coverage, 
percent  mission  coverage  by  UAV  type,  and  the  mission  value  generated  for  each  run. 
The  data  required  to  calculate  these  MOEs  reside  in  two  tables:  Coverage  and 
CoverageByType.  Since  the  Coverage  table  consisted  of  21,360  rows,  we  converted  the 
table  to  Excel®  format.  We  created  Visual  Basic®  macros  to  calculate  the  percent 
mission  type  coverage  for  all  mission  payloads  and  the  mission  value  for  each  design 
point.  Using  the  same  set  of  macros,  we  extracted  the  MOEs  and  consolidated  them  with 
the  original  design  spreadsheet. 

The  CoverageByType  consists  of  106,800  rows  and  is  therefore  too  large  to  save 
as  an  Excel®  file.  A  short  application  was  created  in  Java™  using  the  Java  Database 
Connectivity  (JDBC)  to  access  the  CoverageByType  table  within  Microsoft®  Access®, 
calculate  the  percent  mission  coverage  by  UAV  type,  and  output  the  results  as  a  comma 
separated  value  (.csv)  file.  The  Java™  code  is  displayed  in  Appendix  E.  The  results  were 
merged  with  the  design  matrix,  creating  a  row  for  each  design  point  with  20  additional 
columns  for  the  MOEs. 

2.  JMP  Statistical  Discovery  Software™ 

In  order  to  facilitate  our  analysis,  we  selected  JMP  Statistical  Discovery 
Software™  from  SAS.  JMP  Software  provides  a  stand-alone  analytical  tool  for  DOE 
development,  implementation,  visualization  and  analysis.  The  research  analyst  can  learn 
the  fundamentals  within  JMP  in  order  to  input,  manipulate,  and  begin  analysis  of  their 
data  within  a  few  hours.  JMP  provides  the  following  capabilities  that  were  specifically 
useful  during  our  research: 

•  Spreadsheet  environment  for  entering,  editing,  and  displaying  data  sets. 

•  Interactive  graphs  and  reports  that  assist  in  the  exploration  of  the  data. 

•  Journaling  tools  that  record  the  investigative  process  for  follow-on  review. 
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In  closing,  a  space-filling  design  coupled  with  a  robust  software  package  assisted 
our  exploration  of  the  response  surface  represented  by  the  multiple  Measures  of 
Effectiveness  (MOE)  depicted  in  the  solution  of  ASC-U. 
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V.  DATA  ANALYSIS 


This  chapter  describes  the  analysis  of  our  data.  Section  A  provides  an  overview 
of  the  techniques  used  during  the  analysis.  In  Section  B,  we  provide  an  assessment  of  our 
first  design  and  the  data  characteristics  that  led  to  the  development  of  a  new  design.  We 
continue  with  initial  observations  of  the  data  generated  by  the  new  design  using 
histograms  and  a  scatter  plot  of  the  mission  value  by  design  points.  Section  C  provides 
an  in-depth  analysis  of  the  regression  model  using  mission  value  as  the  response  variable. 
Section  D  develops  an  alternate  regression  model  using  percent  mission  coverage  as  the 
response  variable.  In  Section  E,  we  describe  interesting  observations  by  comparing  the 
measures  of  effectiveness  used  in  our  two  models  and  provide  an  interpretation  of  our 
model  coefficients. 

A.  MULTIPLE  REGRESSION  ANALYSIS 

Multiple  regression  analysis  creates  a  mathematical  description  of  the  relationship 
between  a  dependent  variable  (y)  and  several  independent  variables  (x)  (Devore,  2004). 
We  call  the  dependent  variable  the  response  and  the  independent  variables  regressors. 
The  technique  involves  predicting  (fitting)  the  response  variable  using  a  linear 
combination  of  the  regressors.  The  linear  model  is  described  in  two  forms,  the  true 
model  and  the  estimated  model.  The  true  model  is  a  representation  of  the  actual  observed 
values.  The  estimated  model  represents  the  predicted  or  expected  values.  We  enumerate 
the  regressors  as  x/,  X2,  . . .,  .vt,  where  k  represents  the  number  of  regressors  (k  >=  2).  The 
model  also  includes  parameters  fi  and  s.  The  parameters [h,  [h,  ...,/4  are  coefficients  that 
represent  the  amount  the  response  variable  changes  when  the  corresponding  regressor 
changes  by  one  unit.  The  intercept,  /?&  is  a  constant  where  the  regression  line  intercepts 
the  y-axis.  Error  in  the  true  model  is  represented  by  e.  It  is  the  difference  between  the 
actual  value  and  its  predicted  value.  In  the  estimated  model,  the  difference  is  called  a 
residual  (Sail,  Creighton,  &  Lehman,  2005).  Figure  16  displays  the  components  of  a 
general  linear  model  with  two  terms. 
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I  1 


t 
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1 


y  =  Po  +  Pixi  +  p2x2  +  s 


y  =  bo  +  bjXi  +  b2X2  +  e 

ft  t  t 

parameter  estimates 

residual 


s  ~  Normal(0,  a2) 
a2  is  estimated  by  s2 
SSE  =  YJei 


Figure  16.  Components  of  a  General  Linear  Model  (After  Sail  et  al.,  2005). 


Figure  16  represents  a  first-order  model.  However,  variations  of  the  regression 
model  may  also  include  second-order  terms  (polynomial)  to  investigate  curvilinear 
effects,  cross-product  terms  (factorial)  to  investigate  interaction  effects,  and  full  quadratic 
models  that  combine  both  second-order  and  factorial  terms  in  one  equation.  Creating  full 
quadratic  models  during  regression  analysis  may  result  in  a  robust  model  that  is  able  to 
explain  a  larger  portion  of  the  variation  in  the  response  variable  compared  to  a  simple 
first-order  model.  Figure  17  displays  four  multiple  regression  models  using  two 
independent  variables. 


First-Order  Model 

Y  =  ft0+fiiXi  +  p2xi  +  £ 

Second-Order  Model 

Y  =  ft 0  +  fi\X\  +  P2X2  +  /?3*2  +  P4X  2  +  £ 

First-Order  with  Interactions 

Y  =  0  +  P\X\  +  P2X2  +  ft 3X1X2  +  £ 

Full  Quadratic  Model 

Y  =  Po  +  P1X1+  P2X2+  PiX2i+  ^4x1  +  P5X1X2+  e 

Figure  17.  Multiple  Regression  Models. 
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To  conduct  meaningful  regression  analysis,  we  must  verify  certain  assumptions 
about  the  data  set.  First,  there  must  be  at  least  an  approximate  linear  relationship 
between  the  response  variable  and  the  regressors.  Analysts  check  this  relationship  by 
overlaying  a  graph  of  the  regression  equation  on  the  scatter  plot  of  the  data  (Devore, 
2004).  Next,  the  residuals  must  have  a  zero  mean  and  constant  variance.  Finally,  the 
residuals  must  be  normally  distributed.  Analysts  can  use  histograms  and  Normal  quantile 
plots  to  check  the  Normality  assumption.  JMP  uses  the  Shapiro-Wilk  test  ( W-statistic )  to 
test  Normality  when  n  <=  2000  (Sail  et  al.,  2005). 

The  JMP  statistical  software  package  reports  the  /-ratio  and  corresponding 
P-value  for  each  regressor  in  the  model.  JMP  uses  the  /-ratio  to  test  the  contribution  of 
each  regressor.  The  procedure  tests  the  significance  of  an  individual  regression 
coefficient,  /?,.  The  null  hypothesis  Hq.  fij  =  0  is  rejected  over  the  alternate  hypothesis 
H\\  0  '^\t0\>tal2n_k_l.  If  the  alternate  hypothesis  is  accepted,  then  we  conclude  that 

the  regressor  xj  is  significant  given  the  other  regressors  remain  in  the  model. 
(Montgomery  et  al.,  2001)  The  /-statistic  is  calculated  using  the  standard  error  for  the 

slope,  s e(/?;): 


°  se{fit) 

Multiple  regression  analysis  allows  the  analyst  to  identify  a  set  of  regressors  that 
explain  a  proportion  of  the  variance  in  the  response  variable.  The  level  at  which  the 
regressors  explain  the  variance  of  the  model  is  established  with  a  significance  test  of  R- 
square  ( R  ).  R-square  is  calculated  using  the  sum  of  squared  residuals  (SSE)  and  the  total 
sum  of  squares  (SST): 


R2 


SSE 

SST 


The  analyst  can  use  an  Adjusted  R-square  ( R2 )  to  compare  models  with  different 

numbers  of  parameters  ( k ).  The  Adjusted  R-square  is  calculated  using  the  mean  squared 
error  [  MSE  =  SSE/(n-(k+l))  ]  and  the  mean  square  of  treatments  [  MST  =  SST/(n-l)  ]: 
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R:  =  I 


MSE 

MST 


Another  method  used  to  compare  regression  models  is  called  the  F  test.  The 
analyst  builds  two  models  consisting  of  different  numbers  of  regressor  variables.  The  full 
model  contains  more  regressors  (k)  than  the  reduced  model  with  l  regressors.  Since  the 
full  model  consists  of  additional  regressors,  it  should  fit  the  data  as  well  or  better  than  the 
reduced  model.  The  F  test  uses  the  sum  of  squared  residuals  for  the  full  model  (SSE/,) 
and  the  reduced  model  (SSE/)  to  calculate  the  value  of  the  /-statistic.  The  full  model  is 
rejected  over  the  reduced  model  if/  >  Fak_ln_(k+X)  (Devore,  2004). 


(SSE,-SSEk)/(k-l) 
SSEk/[n-(k  + 1)] 


We  used  stepwise  regression  to  assist  in  model  development  due  to  the  number  of 
regressors  in  our  study.  JMP  allows  the  analyst  to  select  between  backward  elimination, 
forward  selection  and  mixed  stepwise  regression.  We  selected  mixed  stepwise  regression 
for  our  analysis.  The  procedure  begins  with  forward  selection  by  adding  variables  one  at 
a  time  based  on  the  one  that  generates  the  largest  absolute  /-ratio  when  it  enters  the 
model.  After  adding  a  regressor  to  the  model,  it  examines  the  previously  entered 
variables  for  the  smallest  absolute  /-ratio.  JMP  compares  the  corresponding  E-value  for 
the  candidate  /-ratios  to  user-defined  probabilities  to  enter  or  leave  the  model,  and 
processes  them  respectively.  The  process  stops  when  variables  are  no  longer  significant 
(Sail  et  al.,  2005). 
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B.  INITIAL  OBSERVATIONS 

Our  first  design  consisted  of  26  factors  resulting  in  a  257  x  26  design  matrix.  In 
this  design,  the  optimization  interval  contained  ten  levels  from  one  to  ten  simulation 
hours.  Prior  to  running  our  design,  we  predicted  that  the  optimization  interval  would 
have  a  large  impact  on  the  overall  mission  value  and  mission  percent  coverage.  We 
believed  that  the  smaller  the  interval,  the  larger  the  mission  value  generated  with  all  other 
factors  remaining  constant.  We  based  our  prediction  on  the  assignment  process  within 
ASC-U.  The  more  frequently  ASC-U  initiates  an  optimization  sequence,  the  less  it 
misses  “windows  of  opportunity”  for  missions  that  become  available.  Thus,  the 
simulation  is  able  to  reallocate  UAVs  to  gain  a  higher  mission  value  and  increase  the 
percent  mission  coverage.  However,  because  shorter  optimization  intervals  require  longer 
ASC-U  run  times,  we  were  interested  in  finding  out  how  rapidly  the  solution  degraded 
when  the  optimization  interval  increased. 

We  started  our  analysis  by  fitting  the  main  effects  using  the  stepwise  regression 
feature  in  JMP.  The  direction  of  the  regression  was  set  to  “mixed”  with  “combined” 
rules  so  that  the  regressors  that  entered  into  the  model  on  previous  steps  are  reassessed  in 
subsequent  steps.  We  set  the  probability  to  enter  and  leave  at  0.050  and  0.100, 
respectively.  Figure  18  displays  the  actual  value  by  predicted  plot  and  summary  statistics 
of  the  results. 
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Figure  18.  Actual  Value  by  Predicted  Plot  and  Statistics  for  Initial  DOE. 
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The  stepwise  process  selected  eleven  factors:  Optlnt,  EHoriz,  EOprT,  lOprT, 
2Horiz,  20prT,  30prT,  3Radius,  3Trans,  40prT,  and  4Radius.  We  continued  with  a 
standard  least  squares  regression  on  the  eleven  main  effects. 

To  our  surprise,  a  simple  regression  of  11  main  effects  explained  98%  of  the 
variance  within  the  model.  We  were  eager  to  investigate  the  significance  of  the  factors 
using  the  visual  screening  tool  provided  by  JMP  called  the  Prediction  Profiler.  The 
Prediction  Profiler  allows  the  analyst  to  explore  the  effect  on  the  predicted  response 
variable  by  changing  one  or  more  factor  settings  (Sail  et  al.,  2005).  The  importance  of 
the  optimization  interval  was  evident  upon  viewing  the  plots  displayed  in  Figure  19. 
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Figure  19.  Prediction  Profiler  for  Initial  DOE. 


In  Figure  19,  we  see  that  the  optimization  interval  dominates  the  other  factors 
with  its  effect  on  the  response.  While  this  initial  finding  confirmed  our  earlier  prediction, 
we  quickly  realized  that  the  optimization  intervaFs  strong  effect  was  going  to  limit  our 
ability  to  assess  the  other  factors’  influence  on  the  model.  Prior  to  considering  alternate 
options  for  our  analysis,  we  ran  a  simple  regression  using  the  optimization  interval  as  the 
only  regressor  in  order  to  find  out  how  much  of  the  model  it  explained.  Figure  20 
displays  the  results  of  the  one-term  model.  The  regressor  accounted  for  94%  of  the 
variance  within  the  model. 
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Figure  20.  Actual  by  Predicted  Mission  Value  Plot  for  Initial  DOE. 


Not  only  does  optimization  interval  explain  “almost  all”  of  the  variation,  but  the 
mission  values  for  larger  optimization  intervals  show  a  reduction  in  the  overall  mission 
value  by  nearly  50%.  This  is  clearly  unacceptable.  The  practical  implication  is  that  for 
this  scenario,  ASC-U  should  be  used  with  a  short  optimization  interval  in  order  to  obtain 
a  good  solution,  even  though  this  requires  more  computing  time. 

The  dominant  influence  of  the  optimization  interval  led  us  to  explore  other 
options  for  our  analysis.  Our  goal  was  to  eliminate  the  presence  of  the  interval’s  effect 
and  examine  the  other  twenty- five  factors.  We  looked  at  our  design  of  experiment  to  see 
how  many  runs  consisted  of  the  optimization  interval  set  to  one.  There  were  fifteen 
instances,  not  enough  for  the  number  of  predictor  variables  in  our  study.  Tabachnick  and 
Fidell  (1989)  propose  the  rule  of  thumb  for  testing  R-square  using  N  >=  50  +  8m,  where 
N  =  number  of  observations  and  m  =  number  of  predictors.  Since  we  were  interested  in 
the  effects  of  the  twenty-five  UAV  factors,  we  needed  a  sample  size  nearly  the  size  of  our 
original  design.  Otherwise,  a  situation  where  m  >=  N  results  in  a  trivial  solution  of 
R-square  =  1.0. 
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We  generated  a  new  design  of  experiment  using  the  twenty-five  UAV  factors. 
We  concatenated  the  15  runs  from  the  original  design  with  the  257  new  runs.  The  result 
was  a  maximum  pairwise  correlation  of  0.0372,  providing  nearly  orthogonal  properties 
for  our  design  (Cioppa,  2002).  We  generated  the  new  data  set  and  combined  the  results 
with  the  original  fifteen  runs,  giving  us  272  observations  for  our  analysis. 

Once  we  completed  the  runs  for  the  new  design,  we  computed  the  mission  values 
and  twenty  MOEs.  We  consolidated  the  results  with  their  corresponding  design  points 
using  the  methodology  described  in  Chapter  IV.  Our  initial  assessment  consisted  of 
reviewing  the  distributions  of  the  MOEs  as  displayed  in  Figures  21-23.  Figure  21 
displays  the  distribution  and  summary  statistics  of  the  mission  value  for  all  272  designs. 
The  base  scenario  design  generated  a  mission  value  of  4224154,  slightly  above  the  mean 
of  4215522.  Many  of  the  MOEs  are  approximated  by  a  normal  distribution,  with  the  base 
scenario  falling  in  the  center  of  the  data.  However,  several  MOEs  present  a  bimodal 
distribution  that  warrant  further  investigation. 
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Figure  2 1 .  Distribution  and  Summary  Statistics  of  Mission  Value. 
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Figure  22.  Distribution  and  Summary  Statistics  for  MOEs. 
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Figure  23.  Distribution  and  Summary  Statistics  for  MOEs. 
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Of  particular  interest  are  the  percent  coverage  distributions  for  the  following 
payloads:  meteorological  (Meteo),  communication  intelligence  (COMINT),  foliage 
penetrating  radar/light  detection  and  ranging  (FOPEN/LIDAR),  and  supply  delivery 
(Supply).  We  investigated  the  significant  individual  factors  that  influence  the  behavior  of 
the  first  two  MOEs  forming  bimodal  distributions.  We  identify  the  factor(s)  that  tend  to 
cause  the  MOEs  to  concentrate  into  separate  regions.  Our  preliminary  findings  for  Meteo 
and  COMINT  are  displayed  in  Appendices  C  and  D,  respectively.  The  last  two  MOEs 
have  little  to  no  variation  in  their  percent  coverage.  FOPEN/LIDAR  predominantly 
receives  a  percent  coverage  of  46.4%,  whereas  the  solution  does  not  allocate  UAYs  to 
supply  delivery  missions.  We  investigate  FOPEN/LIDAR  coverage  and  supply  delivery 
missions  in  Section  E  of  this  chapter. 

After  reviewing  the  histograms  for  each  MOE,  we  generated  the  scatterplot  for 
mission  value  in  Figure  24.  We  observed  one  design  point,  140,  falling  below  3.13 
standard  deviations  from  the  mean.  Additionally,  we  were  particularly  interested  in  the 
five  extreme  values  located  at  the  top  of  the  plot  achieving  high  overall  mission  values. 
In  the  next  section,  we  discuss  the  regression  analysis  that  investigates  these  observations 
further. 


Figure  24.  Scatterplot  of  Mission  Value  for  Design  Points. 


61 


C.  MISSION  VALUE  MODEL 

This  section  provides  an  analysis  of  the  regression  model  using  mission  value  as 
the  response  variable.  We  were  interested  in  the  effects  that  the  UAV  characteristics  and 
performance  capabilities  had  on  the  ASC-U  generated  mission  value  for  each  scenario. 


1.  Main  Effects  Model 

We  started  our  analysis  with  a  mixed  stepwise  regression  on  the  main  effects 
using  a  probability  to  enter  and  leave  of  0.050  and  0.100  respectively.  The  stepwise 
process  selected  12  of  the  25  regressors:  EHoriz,  EOprT,  lHoriz,  20prT,  IRadius, 
2Radius,  3Horiz,  30prT,  3Radius,  4Horiz,  40prT,  and  4Radius.  We  performed  a 
standard  least  squares  regression  on  the  selection.  Figure  25  displays  the  actual  value  by 
predicted  plot  for  the  regression.  The  model  generated  an  R-square  value  of  0.8895. 
Eight  of  the  twelve  factors  were  significant  with  a  Prob>|t|  <0.0001.  Three  more  were 
significant  at  0.0061  or  less.  One  factor,  2Radius,  generated  a  Prob>|t|  of  0.0265. 


Figure  25.  Actual  by  Predicted  Mission  Value  Plot  for  Main  Effects  Model. 
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2.  First  Order  Model  with  Interactions 

We  would  like  to  increase  the  explanation  of  the  variance  within  the  model  and  at 
the  same  time  keep  the  significant  factors  to  a  minimum.  In  order  to  establish  a  range  of 
R-square  values,  identify  significant  factors,  and  further  investigate  trends  within  the 
data,  we  generated  a  first  order  model  with  interactions.  We  started  with  all  main  effects. 
Next,  we  ran  the  macro  within  JMP  to  add  effects  with  factorial  to  degree  set  at  2.  We 
performed  a  stepwise  regression  followed  by  least  squares  regression  as  described  above. 

Figure  26  displays  the  actual  value  by  predicted  plot  for  the  regression.  Again, 
we  notice  a  high  level  of  stability  between  the  predicted  plot  and  the  actual  data.  The 
stepwise  process  selected  20  main  effects  and  24  interaction  terms.  The  model  generated 
an  R-square  value  of  0.9557.  Nineteen  of  the  factors  were  significant  with  a  Prob>|t| 
<0.0001.  Ten  of  the  factors  generated  a  Prob>|t|  greater  than  0.0100.  Interestingly,  each 
UAY  type  presented  four  of  five  factors  within  the  model.  Air  speed  was  not  selected  for 
ERMP,  Class  II,  and  Class  III  UAVs.  Transition  time  was  not  selected  for  Class  I  and 
Class  II  UAVs.  We  will  investigate  the  selection  of  significant  factors  as  we  reduce  the 
number  of  regressors  to  develop  the  final  model. 


Figure  26.  Actual  by  Predicted  Mission  Value  Plot  for  First  Order  with  Interactions. 
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3.  Full  Quadratic  Model 

At  this  stage,  we  combined  our  model  forms  to  create  a  full  quadratic  model  with 
main  effect,  polynomial,  and  interaction  terms.  We  employed  the  same  techniques  as 
described  above  by  using  JMP  to  add  main  effects  and  interaction  terms.  JMP  also 
provides  a  macro  to  add  polynomial  terms  to  a  user-defined  degree  setting.  We  selected 
polynomial  to  degree  set  at  2.  Prior  to  selecting  stepwise  regression  on  our  model  effects, 
we  changed  the  probability  to  leave  from  0.0100  to  0.0500.  This  was  done  in  order  to 
eliminate  excessive  terms  that  had  a  Prob>|t|  greater  than  0.0100.  Our  first  order 
interaction  model  had  ten  such  effects  that  did  not  significantly  explain  the  model. 

Figure  27  displays  the  actual  value  by  predicted  plot  for  the  full  quadratic  model. 
The  stepwise  process  selected  35  significant  factors  consisting  of  19  main,  1 1  interaction, 
and  5  second-order  terms.  The  model  generated  an  R-square  value  of  0.9610.  Sixteen  of 
the  factors  were  significant  with  a  Prob>|t|  <0.0001.  Fourteen  of  the  factors  were 
significant  at  0.0080  or  less.  Five  of  the  factors  generated  a  Prob>|t|  greater  than  0.0100, 
half  the  amount  found  in  the  first  order  interaction  model. 


Figure  27.  Actual  by  Predicted  Mission  Value  Plot  for  Full  Quadratic  Model. 
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4.  Final  Model 

We  evaluated  the  step  history  generated  in  the  stepwise  regression  of  the  full 
quadratic  model  in  order  to  construct  the  final  model  consisting  of  nine  main  effects,  one 
interaction  term,  and  one  second-order  polynomial  term.  The  statistical  report  for  the 
regression  model  for  mission  value  is  displayed  in  Appendix  A.  JMP  provides  the  R- 
square  values  produced  by  the  remaining  terms  in  the  model  at  each  iteration  of  the 
stepwise  process.  Table  9  displays  the  significant  model  terms  and  corresponding  R- 
square  values.  Next,  we  evaluated  the  R-square  values  at  the  “knee  in  the  curve” 
displayed  in  Figure  28  using  the  F  test  described  previously  in  this  chapter. 


Terms 

Parameter 

R-Square 

1 

4Radius 

0.52544 

2 

lOprT 

0.67606 

3 

EHoriz 

0.76483 

4 

EOprT 

0.81134 

5 

(EHoriz-35.9408)*(EOprT-35.9651) 

0.82657 

6 

3  Radius 

0.84766 

7 

1  Radius 

0.86559 

8 

(4Radius-75 ,0007)*(4Radius-75 .0007) 

0.89281 

9 

4Horiz 

0.90568 

10 

3Horiz 

0.91541 

11 

40prT 

0.92098 

12 

lHoriz 

0.92458 

13 

30prT 

0.92889 

14 

(3Horiz-6.00654)*(30prT-5 .98 1 1 8) 

0.93172 

15 

(EOprT-3  5.965 1  )*(EOprT-3  5 .965 1 ) 

0.93652 

16 

1  Speed 

0.93808 

17 

2Radius 

0.94023 

18 

20prT 

0.94130 

19 

(20prT-1.99919)*(40prT-6.00765) 

0.94293 

20 

(EHoriz-3  5 . 9408)  *  (4Radius-7 5.0007) 

0.94445 

Table  9.  Significant  Model  Terms  and  Corresponding  R-square  Values  for  Mission 

Value  Model. 
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Model  Terms 


Figure  28.  R-Square  Value  and  Corresponding  Terms  from  Stepwise  History  for 

Mission  Value  Model. 


Our  goal  was  to  maintain  a  meaningful  explanation  of  the  variance  within  the 
model  and  keep  the  significant  factors  to  a  minimum.  While  R 2  provides  a  measure  of 
the  explained  variance  within  a  given  model,  it  can  be  misleading  when  the  model 
contains  many  regressors  compared  to  the  size  of  the  sample  (Devore,  2004).  We 
explored  the  addition  of  each  term  to  the  model  using  the  model  utility  F  test.  With  the 
F  test,  we  can  develop  an  assessment  as  to  whether  additional  predictors  provide  a 
significant  increase  to  the  explanation  of  the  model  (Devore,  2004). 

Table  10  displays  the  sequential  F  test  for  the  full  model  containing  all  twenty 
remaining  terms  from  the  stepwise  history.  The  right-hand  side  of  the  table  displays  the 
critical  values  for  the  F  Distribution  for  a  test  with  a  set  at  0.0001,  0.0010,  0.0100,  and 
0.0500  for  comparison.  For  the  study,  we  were  interested  in  the  level  of  significance  with 
a  =  0.0001.  As  we  move  down  the  table,  we  are  adding  predictors  and  interested  in  the 
case  where  the  /-statistic  is  less  than  the  F  critical  value  for  the  first  time.  For  our  model, 
this  occurs  when  adding  the  twelfth  regressor.  We  conclude  that  the  addition  of  the  first 
eleven  regressors  contribute  significantly  to  the  model. 
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n  = 

272 

numerator 

denominator 

|  F  a,k  -l,n-  (k+  1)  | 

Terms 

SSE 

df 

df 

/ 

0.0001 

0.0010 

0.0100 

0.0500 

1 

7.49E+11 

2 

5.11  E+ 1 1 

1 

269 

125.07435 

15.6003 

11.0693 

6.7300 

3.8763 

3 

3.71  E+ 1 1 

1 

268 

101.16488 

15.6021 

11.0702 

6.7304 

3.8764 

4 

2.98E+11 

1 

267 

65.80961 

15.6039 

11.0712 

6.7307 

3.8765 

5 

2.74E+11 

1 

266 

23.36293 

15.6057 

11.0721 

6.7311 

3.8767 

6 

2.40E+1 1 

1 

265 

36.69850 

15.6075 

11.0730 

6.7315 

3.8768 

7 

2.12E+11 

1 

264 

35.20091 

15.6093 

11.0740 

6.7318 

3.8769 

8 

1.69E+11 

1 

263 

66.79908 

15.6111 

11.0749 

6.7322 

3.8771 

9 

1.49E+11 

1 

262 

35.74072 

15.6130 

11.0759 

6.7326 

3.8772 

10 

1.33E+11 

1 

261 

30.03970 

15.6149 

11.0769 

6.7330 

3.8773 

11 

1.25E+11 

1 

260 

18.30804 

15.6167 

11.0778 

6.7334 

3.8775 

12 

1.19E+11 

1 

259 

12.35303 

15.6186 

11.0788 

6.7337 

3.8776 

13 

1.12E+11 

1 

258 

15.66285 

15.6205 

11.0798 

6.7341 

3.8778 

14 

1.08E+11 

1 

257 

10.63874 

15.6225 

11.0808 

6.7345 

3.8779 

15 

1.00E+11 

1 

256 

19.35920 

15.6244 

11.0818 

6.7349 

3.8780 

16 

9.77E+10 

1 

255 

6.43972 

15.6264 

11.0828 

6.7353 

3.8782 

17 

9.43E+10 

1 

254 

9.12737 

15.6283 

11.0838 

6.7357 

3.8783 

18 

9.26E+10 

1 

253 

4.61943 

15.6303 

11.0849 

6.7361 

3.8785 

19 

9.00E+10 

1 

252 

7.18978 

15.6323 

11.0859 

6.7365 

3.8786 

20 

8.76E+10 

1 

251 

6.87798 

15.6343 

11.0870 

6.7369 

3.8788 

Table  10.  Model  Utility  F-test  for  Twenty  Terms  Selected  in  the  Stepwise 
Regression  for  the  Mission  Value  Model. 


Figure  29  displays  the  actual  value  by  predicted  plot  for  the  final  model  consisting 
of  9  main  effects,  1  two-way  interaction,  and  1  second-order  polynomial.  The  eleven 
terms  account  for  92%  of  the  variance  within  the  model.  Two  predictors  accounted  for 
nearly  68%  of  the  variance  within  the  model:  Class  IV  operating  radius  at  53%,  and 
Class  I  operating  time  for  another  15%.  All  of  the  factors  were  significant  with  a 
Prob>|t|  <0.0001.  Before  exploring  the  significant  factors  and  interactions,  we  verified 
the  model’s  validity. 

Recall  that  the  model’s  summary  statistics  such  as  the  t  and  F  ratios  as  well  as  the 
R  value  depend  on  several  assumptions.  We  checked  the  validity  of  the  assumptions  by 
first  investigating  the  actual  value  by  predicted  plot.  Figure  29  indicates  that  the 
relationship  between  the  response  variable  and  the  regressors  is  linear.  Next,  we 
confirmed  that  the  residuals  displayed  a  mean  of  zero  and  constant  variance  by  observing 
the  residual  by  predicted  value  plot  in  Figure  30.  Finally,  we  checked  the  normality 
assumption  with  a  histogram  and  Normal  quantile  plot  of  the  residuals  displayed  in 
Figure  31.  The  plots,  along  with  a  Shapiro-Wilk  value  of  0.99  at  a  significance  of  0.46, 

indicate  that  we  can  safely  assume  the  residuals  meet  the  Normality  assumption. 
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Figure  29.  Actual  by  Predicted  Value  Plot  for  Final  Mission  Value  Model. 


Figure  30.  Residual  by  Predicted  Value  Plot  for  Final  Mission  Value  Model. 
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Figure  3 1 .  Histogram  and  Normal  Quantile  Plot  of  Residuals  for  Final  Mission  Value 

Model. 


5.  Significant  Factors 

As  mentioned  in  the  previous  section,  Class  IV  operating  radius  and  Class  I 
operating  time  explained  68%  of  the  variation  in  the  mission  value.  In  Section  B,  we  saw 
that  the  optimization  interval  dominated  the  other  factors  with  its  effect  on  the  response. 
Our  new  design,  with  the  optimization  interval  set  at  one,  presents  significant  findings 
involving  other  factors.  The  Prediction  Profiler  in  Figure  32  indicates  that  Class  IV 
operating  radius  has  the  strongest  effect  on  mission  value.  Class  I  operating  time,  ERMP 
time  horizon  and  operating  time  provide  a  lesser  effect.  The  remaining  main  effects 
demonstrate  a  weak  effect  on  the  response. 

The  second-order  polynomial  term  for  Class  IV  operating  radius  also  introduces  a 
curvilinear  effect.  While  an  increase  in  the  Class  IV  operating  radius  tends  to  increase 
the  overall  mission  value,  there  is  a  point  of  diminishing  returns.  We  also  note  that  there 
is  a  direct  relationship  between  Class  IV  operating  radius  and  mission  value.  The  same 
trend  occurs  for  Class  I  operating  time  and  mission  value.  However,  ERMP  time  horizon 
demonstrates  an  inverse  relationship  with  mission  value.  We  explore  this 
counterintuitive  effect  later  in  this  section. 
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Figure  32.  Prediction  Profiler  for  Final  Mission  Value  Model. 


We  explored  the  significant  factors  further  using  JMP’s  partitioning  platform. 
JMP  allows  the  analyst  to  generate  regression  trees  as  a  method  of  exploratory  modeling. 
Regression  trees  employ  a  binning  and  averaging  process.  The  software’s  algorithm 
evaluates  all  of  the  predictor  values  in  order  to  determine  the  optimum  split  in  the  tree. 
The  predictor  value  that  generates  the  highest  reduction  in  total  sum  of  squares  is  selected 
to  create  the  branch  in  the  tree  (Sail  et  al.,  2004). 

Figure  33  displays  the  regression  tree  for  mission  value  after  the  first  two  splits. 
In  the  first  split,  the  tree  indicates  that  when  Class  IV  aircraft  have  an  operating  radius  of 
greater  than  or  equal  to  68.9  km,  the  overall  mission  value  tends  to  be  higher.  In  fact,  the 
mean  of  the  resulting  mission  value  represents  an  increase  by  8.2%  of  the  total  mission 
value  range  generated  by  our  design.  This  also  corresponds  to  a  two  percent  increase 
over  the  mission  value  of  the  baseline  scenario.  The  Class  I  operating  time  variable 
generates  the  second  split.  For  UAV  configurations  where  Class  IV  aircraft  have  an 
operating  radius  above  68.9  km,  Class  I  operating  time  is  the  best  predictor  of  mission 
value.  Class  I  platforms  with  an  operating  time  greater  than  0.901  hrs  increase  the  mean 
mission  value  by  18.5%  of  the  total  mission  value  range.  The  significance  of  this  finding 
is  that  by  considering  levels  for  only  two  airframe  factors  out  of  twenty-five,  we  can 
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increase  the  mean  of  the  mission  value  by  nearly  19%.  Additionally,  an  increase  in  the 
Class  I  UAV’s  operating  time  by  at  least  4.26  minutes  (8.5%  of  the  base  scenario  setting) 
accounted  for  more  than  a  ten  percent  increase  to  the  mean  mission  value.  Perhaps  even 
more  interesting,  it  appears  that  improvements  to  the  mean  mission  value  can  be  achieved 
even  if  the  Class  IV  UAV  operates  with  a  maximum  radius  that  is  6.1  km  smaller  than 
established  by  the  base  scenario  setting. 


Figure  33.  Regression  Tree  for  Mission  Value  Indicating  the  Significance  of  Class  IV 
Operating  Radius  and  Class  I  Operating  Time. 
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From  the  regression  analysis  results,  we  observed  the  significance  of  the  Class  IV 
operating  radius  and  Class  I  operating  time.  Additionally,  the  regression  tree  in  Figure  33 
provided  an  assessment  in  terms  of  the  significant  levels  for  each  factor.  Next,  we 
produced  a  contour  plot  to  demonstrate  how  the  two  factors  affect  the  mission  value 
together.  Figure  34  displays  the  contour  plot  for  Class  IV  operating  radius  vs.  Class  I 
operating  time.  Filled  contour  regions  depict  the  mission  value  as  the  radius  and 
operating  time  varies.  Red  regions  in  the  lower  edge  of  the  plot  indicate  mission  values 
less  than  or  equal  to  4.0  x  106.  Blue  and  purple  regions  in  the  top  and  upper-right  depict 
mission  values  greater  than  4.3  x  106.  The  plot  indicates  that  above  average  mission 
values  can  be  achieved  with  a  large  range  of  settings  as  indicated  in  the  region  above 
line  1.  We  also  get  a  better  feel  for  the  diminishing  returns  on  increased  Class  IV 
operating  radius  as  the  Class  I  operating  time  is  changed.  Line  2  indicates  the  point  at 
which  substantial  enhancements  to  mission  value  by  Class  IV  operating  radius  begin  to 
level-off.  This  effect  varies  across  a  wide  range  of  Class  I  operating  time  mission  values. 


Contour  Plot  for  Value 


— Legend- 
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Class  I  Operating  Time 


I  <=  4.0000e+6 
I  <=  4.0500e+6 
I  <=  4.1000e+6 
I  <=4.1500e+6 
I  <=  4.2000e+6 
I  <=  4.2500e+6 
I  <=  4.3000e+6 
I  <=  4.3500e+6 
I  >  4.3500e+6 


Figure  34.  Contour  Plot  for  Mission  Value  with  Class  IV  Operating  Radius  vs.  Class 

I  Operating  Time. 
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The  next  section  explores  the  significant  interactions  observed  within  the  model. 
Our  discussion  makes  use  of  interaction  plots  to  investigate  the  effect  between  ERMP 
time  horizon  and  operating  time.  The  plots  further  highlight  the  diminishing  return  on 
mission  value  due  to  increased  operating  radius  of  Class  IV  aircraft. 


6.  Significant  Interactions 

Our  final  model  for  mission  value  contained  one  interaction  term,  ERMP  time 
horizon  with  ERMP  operating  time.  Recall  from  Chapter  II  that  the  Predator-based 
ERMP  UAV  is  the  largest  of  the  five  UAVs  investigated  in  this  study.  It  serves  as  a 
division  level  asset  and  is  capable  of  serving  all  mission  requirements  except  for  SIGINT 
payloads.  Significant  findings  and  insights  may  assist  in  the  development  of  employment 
considerations  for  this  unmanned  platform. 

Figure  35  displays  the  profile  plot  indicating  interaction  between  the  two  factors. 
In  the  plot,  the  y-axis  represents  the  mission  value.  The  smaller  plots  display  the  effect  of 
two  factors  on  the  response.  Solid  lines  indicate  interaction  between  terms.  There  is  one 
term  listed  on  the  diagonal  plot  for  each  column.  The  x-axis  at  the  bottom  of  the  graph 
represents  the  term’s  range.  Changes  in  the  term’s  value  affect  the  mission  value 
according  to  the  slope  of  the  lines  found  in  the  plots  above  and  below  the  diagonal.  The 
plots  across  each  row  display  two  lines,  one  each  for  the  low  and  high  values  of  the  term 
listed  in  the  diagonal  for  the  given  row.  We  can  assess  changes  in  the  response  variable 
when  the  row  term  is  set  to  its  low  or  high  value  and  we  vary  the  range  of  a  term  found  in 
the  columns  to  the  left  or  right  of  the  diagonal,  holding  all  other  effects  constant. 
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Figure  35.  Interaction  Profile  for  Final  Mission  Value  Model. 


Looking  at  the  middle  row,  we  see  ERMP  time  horizon  crossed  with  ERMP 
operating  time  on  the  center-right  plot.  The  top  line  represents  the  change  to  mission 
value  when  the  time  horizon  is  set  at  28.8  hrs.  The  fact  that  this  line  is  nearly  flat  with  a 
slight  positive  slope  indicates  that  there  is  only  a  small  increase  in  mission  value  as  the 
ERMP  operating  time  increases  across  its  entire  range.  However,  when  ERMP  time 
horizon  is  set  to  its  highest  level  of  43.2  hrs,  mission  value  increases  substantially  as 
ERMP  operating  time  increases.  For  our  current  range  of  values,  the  red  line  (top) 
represents  the  highest  mission  values  when  the  horizon  time  is  set  to  the  ERMP  UAV’s 
lowest  level  of  28.8  km.  This  finding  seems  counterintuitive.  It  implies  that  when 
planning  for  ERMP  UAV  missions  we  should  be  myopic  in  our  approach  and  consider  a 
set  of  mission  areas  that  become  available  sooner  rather  than  a  larger  set  of  potential 
mission  areas  over  a  broader  time  window.  Upon  further  investigation  of  the  base 
scenario,  we  identified  an  explanation  as  to  why  the  solution  behaves  in  this  manner. 

The  scenario  examined  in  this  study  involves  21,360  mission  requirements,  of 

which  21,317  represent  missions  with  a  time  window  of  28.8  hrs  or  less.  If  the  ERMP 
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time  horizon  is  greater  than  28.8,  the  optimization  process  attempts  to  allocate  the  UAV 
resource  to  missions  of  greater  value  within  the  extended  future  time  horizon.  In  doing 
so,  the  process  skips  available  missions  of  lesser  value  that  are  closer  to  the  current 
simulation  time.  The  cumulative  and  potentially  larger  value  generated  from  serving 
multiple  missions  is  lost.  Additionally,  when  the  ERMP  time  horizon  is  set  to  28.8  hrs, 
the  mission  value  increase  only  slightly  as  the  operating  time  is  increased.  Based  on  the 
reasoning  above,  this  result  also  makes  sense.  The  lowest  level  for  ERMP  operating  time 
is  28.8  hrs,  identical  to  the  platform’s  lowest  level  for  time  horizon.  The  UAV  only 
requires  28.8  hrs  to  serve  99.8%  of  the  missions  within  the  scenario.  Increases  in  its 
operating  time  may  serve  to  increase  its  available  loiter  time  on  target.  However,  the 
increased  loiter  time  is  negligible  due  to  the  average  mission  length  of  2.24  hrs  for  the 
21,317  missions. 

Additional  insights  gained  from  Figure  35  involve  the  combined  effects  of  Class 
IV  operating  radius  and  ERMP  time  horizon.  The  center-top  plot  indicates  that  relatively 
high  mission  values  are  obtained  when  the  Class  IV  operating  radius  is  set  at  its 
maximum  level  of  90  km  and  ERMP  time  horizon  is  set  at  its  lowest  level  of  28.8  hrs. 
However,  we  also  observe  a  point  of  diminishing  return  for  the  effect  of  Class  IV 
operating  radius  on  mission  value.  The  center-left  plot  on  the  interaction  profde  indicates 
this  effect.  When  ERMP  time  horizon  is  set  to  its  lowest  level  of  28.8  hrs,  the  effect  of 
Class  IV  operating  radius  on  mission  value  diminishes  around  85  km,  when  all  other 
effects  remain  constant.  The  contour  plot  in  Figure  34  also  suggests  high  mission  values 
around  this  range.  We  observe  that  when  Class  I  operating  time  is  as  low  as  0.794  hrs 
and  Class  IV  operating  radius  is  around  86.5  km,  the  simulation  generates  the  sixth 
highest  mission  value,  increasing  by  21%  of  its  entire  range. 

The  insights  gained  from  exploring  the  interaction  between  ERMP  time  horizon 
and  operating  time,  Class  IV  operating  radius,  and  Class  I  operating  time  are  significant. 
In  the  scenario  examined  by  our  study,  the  results  indicate  that  overall  mission  success 
can  be  exploited  by  a  relatively  small  ERMP  time  horizon  and  platform  operating  time. 
The  next  section  develops  an  alternate  regression  model  using  percent  mission  coverage 
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as  the  response  variable.  We  will  discuss  the  development  process,  parameter  estimates, 
and  significant  factors  of  the  model. 


D.  PERCENT  MISSION  COVERAGE  MODEL 

The  regression  model  developed  for  mission  value  provides  an  initial  assessment 
to  the  solution  of  ASC-U.  However,  it  is  difficult  to  relate  mission  value  to  a  tactical 
measure  of  effectiveness.  This  section  describes  the  development  of  a  regression  model 
for  the  overall  percent  mission  coverage  for  the  scenario.  We  begin  with  some  initial 
observations  by  reviewing  the  distribution  of  percent  mission  coverage  and  a  scatterplot 
for  the  data.  The  section  continues  with  a  description  of  the  final  model  and  exploration 
of  significant  factors. 

Figure  36  displays  the  distribution  and  summary  statistics  of  percent  mission 
coverage  for  all  272  designs.  The  base  scenario  design  generated  a  percent  mission 
coverage  of  82.79%,  slightly  above  the  mean  of  82.43%.  A  Shapiro-Wilk  value  of  0.99  at 
a  significance  of  0.97  indicates  that  a  normal  distribution  fits  the  data  well. 


Figure  36.  Distribution  and  Summary  Statistics  of  Percent  Mission  Coverage. 
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After  reviewing  the  summary  statistics,  we  generated  the  scatterplot  for  percent 
mission  coverage  in  Figure  37.  We  observe  similar  extreme  points  as  identified  in  Figure 
21  for  mission  value.  Interestingly,  two  design  points,  21  and  22,  replace  43  and  85  for 
fourth  and  fifth  highest  percent  mission  coverage  values  respectively.  This  finding 
indicates  that  mission  value  does  not  necessarily  depict  equivalent  percent  coverage.  In 
the  next  section,  we  discuss  the  regression  analysis  that  investigates  these  observations 
further. 
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Figure  37.  Scatterplot  of  Percent  Mission  Coverage  for  Design  Points. 

1.  Final  Model 

Our  analysis  followed  the  same  path  as  the  development  process  for  the  mission 
value  model  described  in  Section  C  of  this  chapter.  We  explored  the  main  effects,  first 
order  interactions,  and  second-order  polynomial  terms  using  stepwise  mixed  selection 
and  standard  least  squares  regression.  We  evaluated  the  history  generated  in  the  stepwise 
regression  of  the  full  quadratic  model  in  order  to  construct  a  final  model  consisting  of  ten 
main  effects  and  one  second-order  polynomial.  The  statistical  report  for  the  regression 
model  for  percent  mission  coverage  is  displayed  in  Appendix  B.  Table  1 1  displays  the 
significant  model  terms  and  corresponding  R-square  values.  Next,  we  evaluated  the 
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terms  using  the  F  test.  Table  12  displays  the  sequential  F  test  for  twenty-nine  remaining 
terms  from  the  full  model.  Based  on  the  small  /-statistic  for  the  seventh  term,  we 
removed  it  from  our  final  model.  While  terms  8  through  14  proved  significant  to  the 
model,  we  decided  to  keep  terms  8  through  12  in  an  attempt  to  avoid  overfitting  the 
model  and  maintain  similar  predictive  power  as  our  model  for  mission  value. 


Terms 

Parameter 

R-Square 

i 

lOprT 

0.54420 

2 

4Radius 

0.73984 

3 

1  Radius 

0.79817 

4 

EHoriz 

0.83334 

5 

4Horiz 

0.85283 

6 

3Horiz 

0.86738 

7 

(3Horiz-6.00654)*(4Horiz-6.00434) 

0.87000 

8 

EOprT 

0.88547 

9 

30prT 

0.89683 

10 

(4Radius-7 5 . 0007)  *(4Radius-7 5 . 0007) 

0.90995 

11 

lHoriz 

0.91762 

12 

40prT 

0.92464 

13 

3  Radius 

0.93162 

14 

1  Speed 

0.93574 

15 

20prT 

0.93809 

16 

(20prT- 1. 999 19)*(20prT- 1.999 19) 

0.94353 

17 

(EHoriz-3 5 . 9408)  *  (EOprT-3 5 .9651) 

0.94707 

18 

(3Horiz-6.00654)*(3OprT-5.98 1 1 8) 

0.95030 

19 

2Horiz 

0.95144 

20 

ETrans 

0.95145 

21 

(ETrans-0.49934)*(2Horiz-2.0036) 

0.95289 

22 

(EOprT-35.965  l)*(EOprT-35.965 1) 

0.95450 

23 

4Trans 

0.95239 

24 

2Radius 

0.95350 

25 

(3Radius-40.0978)*(3Radius-40.0978) 

0.95514 

26 

4Speed 

0.95591 

27 

(ETrans-0.49934)*(4Radius-75.0007) 

0.95662 

28 

ESpeed 

0.95698 

29 

(ESpeed-240.495)*(3OprT-5.98 1 1 8) 

0.95791 

Table  1 1 .  Significant  Model  Terms  and  Corresponding  R-square  Values  for  Percent 

Mission  Coverage  Model. 
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n  = 

272 

numerator 

denominator 

F  a,k-l,n-  (k+  1) 

Terms 

SSE 

df 

df 

/ 

0.0001 

0.0010 

0.0100 

0.0500 

1 

0.035946 

2 

0.020517 

1 

269 

202.28733 

15.6003 

11.0693 

6.7300 

3.8763 

3 

0.015917 

1 

268 

77.44711 

15.6021 

11.0702 

6.7304 

3.8764 

4 

0.013143 

1 

267 

56.34660 

15.6039 

11.0712 

6.7307 

3.8765 

5 

0.011606 

1 

266 

35.22608 

15.6057 

11.0721 

6.7311 

3.8767 

6 

0.010459 

1 

265 

29.08245 

15.6075 

11.0730 

6.7315 

3.8768 

7 

0.010252 

1 

264 

5.31781 

15.6093 

11.0740 

6.7318 

3.8769 

8 

0.009032 

1 

263 

35.52874 

15.6111 

11.0749 

6.7322 

3.8771 

9 

0.008136 

1 

262 

28.85256 

15.6130 

11.0759 

6.7326 

3.8772 

10 

0.007102 

1 

261 

38.01830 

15.6149 

11.0769 

6.7330 

3.8773 

11 

0.006496 

1 

260 

24.21723 

15.6167 

11.0778 

6.7334 

3.8775 

12 

0.005943 

1 

259 

24.12402 

15.6186 

11.0788 

6.7337 

3.8776 

13 

0.005392 

1 

258 

26.33869 

15.6205 

11.0798 

6.7341 

3.8778 

14 

0.005067 

1 

257 

16.48008 

15.6225 

11.0808 

6.7345 

3.8779 

15 

0.004882 

1 

256 

9.72016 

15.6244 

11.0818 

6.7349 

3.8780 

16 

0.004453 

1 

255 

24.57174 

15.6264 

11.0828 

6.7353 

3.8782 

17 

0.004174 

1 

254 

16.95005 

15.6283 

11.0838 

6.7357 

3.8783 

18 

0.003920 

1 

253 

16.43830 

15.6303 

11.0849 

6.7361 

3.8785 

19 

0.003830 

1 

252 

5.91667 

15.6323 

11.0859 

6.7365 

3.8786 

20 

0.003829 

1 

251 

0.06687 

15.6343 

11.0870 

6.7369 

3.8788 

21 

0.003755 

1 

250 

4.91819 

15.6364 

11.0880 

6.7373 

3.8789 

22 

0.003667 

1 

249 

5.96095 

15.6384 

11.0891 

6.7378 

3.8791 

23 

0.003538 

1 

248 

9.08561 

15.6405 

11.0902 

6.7382 

3.8792 

24 

0.003477 

1 

247 

4.31076 

15.6426 

11.0912 

6.7386 

3.8794 

25 

0.003421 

1 

246 

4.03289 

15.6447 

11.0923 

6.7390 

3.8795 

26 

0.003393 

1 

245 

2.03795 

15.6468 

11.0934 

6.7394 

3.8797 

27 

0.003320 

1 

244 

5.36951 

15.6489 

11.0945 

6.7399 

3.8799 

28 

0.003231 

1 

243 

6.62582 

15.6511 

11.0956 

6.7403 

3.8800 

29 

0.003164 

1 

242 

5.14026 

15.6533 

11.0968 

6.7408 

3.8802 

Table  12.  Model  Utility  F-test  for  Twenty-nine  Terms  Selected  in  the  Stepwise 
Regression  for  Mission  Percent  Coverage. 

Figure  38  displays  the  actual  by  predicted  percent  mission  coverage  plot  for  the 
final  model.  All  of  the  factors  were  significant  with  a  Prob>|t|  <0.0001.  The  eleven 
terms  account  for  92%  of  the  variance  within  the  model.  Two  predictors  accounted  for 
nearly  74%  of  the  variance  within  the  model:  Class  I  operating  time  at  54%,  and  Class 
IV  operating  radius  for  another  20%.  This  result  was  opposite  that  of  the  mission  value 
model,  which  had  Class  IV  operating  radius  accounting  for  53%  and  Class  I  operating 
time  for  another  15%. 
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Figure  38. 


Figure  39. 


Actual  by  Predicted  Percent  Mission  Coverage  Plot  for  Final  Percent 
Mission  Coverage  Model. 


Residua  t  by  Predicted  Plot 
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Residual  by  Predicted  Percent  Mission  Coverage  Plot  for  Final  Percent 
Mission  Coverage  Model. 
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We  observed  the  residuals  in  Figure  39  with  an  approximate  mean  of  zero  and 
random  presentation,  validating  the  assumption  of  homoscedasticity.  Next,  we  checked 
the  normality  assumption  with  a  histogram  and  Normal  quantile  plot  of  the  residuals 
displayed  in  Figure  40.  The  plots  along  with  a  Shapiro-Wilk  value  of  0.99  at  a 
significance  of  0.26  indicates  the  residuals  meet  the  Normality  assumption. 


Figure  40.  Histogram  and  Normal  Quantile  Plot  of  Residuals  for  Final  Percent 

Mission  Coverage  Model. 


2.  Significant  Factors 

The  percent  mission  coverage  model  contains  a  shift  in  significant  factors 
compared  to  the  mission  value  model  developed  in  Section  C.  We  see  that  the  strength  of 
the  effect  for  Class  I  operating  time  and  Class  IV  operating  radius  have  reversed.  As 
mentioned  in  the  previous  section,  Class  I  operating  time  and  Class  IV  operating  radius 
explained  74%  of  the  model.  The  Prediction  Profiler  in  Figure  41  indicates  that  Class  I 
operating  time  has  the  strongest  effect  on  percent  mission  coverage.  Class  IV  operating 
radius,  Class  I  operating  radius,  and  ERMP  time  horizon  demonstrate  a  lesser  effect.  The 
remaining  main  effects  display  a  relatively  weak  effect  on  percent  coverage. 
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Figure  41 .  Prediction  Profiler  for  Final  Percent  Mission  Coverage  Model. 

As  in  the  mission  value  model,  ERMP  time  horizon  demonstrates  an  inverse 
relationship  with  percent  coverage.  We  also  see  this  effect  with  Class  IV  time  horizon. 
However,  Class  I  time  horizon  matches  our  expectations  of  how  the  size  of  a  “planning 
window”  should  effect  the  overall  mission  coverage  for  a  scenario.  We  can  explain  the 
“myopic”  behavior  for  Class  IV  UAVs  in  a  similar  fashion  as  we  did  for  ERMP  UAVs  in 
Section  C.6.  We  ran  the  experiments  with  the  Class  IV  time  horizon  set  at  4.8  to  7.2  hrs. 
Ninety-four  percent  of  the  missions  within  the  scenario  have  a  length  of  4.8  hours  or  less. 
We  suspect  that  the  UAV  could  complete  many  of  these  missions  prior  to  other  valuable 
missions  becoming  available  in  the  future.  Further  support  for  this  concept  is  the  fact  that 
the  average  mission  length  is  2.24  hrs  for  99.8%  of  the  missions.  As  the  time  horizon 
increases,  the  simulation  attempts  to  allocate  Class  IV  UAVs  to  missions  that  are  more 
valuable  in  the  future.  The  result  is  the  potential  loss  of  missions  that  occur  sooner  and 
hence  the  loss  of  mission  coverage  due  to  a  cumulative  effect. 

We  explain  the  direct  relationship  of  Class  I  time  horizon  to  percent  mission 
coverage  in  a  different  way.  We  ran  the  experiments  with  the  Class  I  time  horizon  set  at 
0.664  to  0.996  hrs.  The  optimization  interval  is  set  to  occur  every  hour.  The  simulation 
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is  only  able  to  allocate  UAVs  to  missions  “seen”  within  the  time  window  for  Class  I 
platforms.  This  results  in  the  simulation  not  being  able  to  “see”  missions  that  occur 
beyond  the  time  of  the  next  optimization  sequence.  Therefore,  Class  I  UAVs  are 
assigned  to  missions  that  are  available  at  the  time  of  the  optimization  sequence  and  up  to 
a  fraction  of  the  time  (0.664-0.996)  until  the  next  optimization.  Since  the  Class  I  also 
varies  its  operating  time  between  0.664  and  0.996  hrs  and  its  transition  time  between 
0.128  and  0.192  hrs,  Class  I  UAVs  that  launch  and  serve  mission  areas  at  each 
optimization  interval  may  not  be  available  for  higher  value  missions  during  the  next 
optimization  sequence.  The  Class  I  UAV  is  unique;  it  is  the  only  platform  with  horizon 
and  operating  times  less  than  the  optimization  interval  of  one.  We  suspect  that  increasing 
the  horizon  and  operating  time  ranges  for  Class  I  platforms  may  yield  interesting  results, 
to  include  curvilinear  effects  where  we  see  a  point  of  diminishing  returns  or  inverse 
effects  on  percent  mission  coverage. 

Figure  42  displays  the  regression  tree  for  percent  mission  coverage  after  the  first 
five  splits.  In  the  first  split,  the  tree  indicates  that  when  Class  I  aircraft  have  an  operating 
time  of  greater  than  or  equal  to  0.787  hrs,  the  overall  percent  coverage  tends  to  be  higher. 
While  this  only  results  in  a  one  percent  increase  to  the  mean  percent  coverage,  it 
represents  nearly  an  eight  percent  increase  over  the  range  of  the  results  for  this  MOE. 
Class  I  operating  time  generates  the  second  split,  increasing  the  mean  by  another  percent 
at  operating  times  greater  than  or  equal  to  0.905  hrs.  The  third  split  falls  under  Class  I 
operating  time  of  less  than  0.787  hrs  and  occurs  when  Class  IV  operating  radius  is 
delineated  above  and  below  69.0  km.  The  fourth  split  occurs  when  Class  I  operating  time 
is  between  0.787  and  0.905  hrs.  The  significant  factor  in  this  split  is  Class  IV  operating 
radius,  and  results  in  an  overall  mean  of  82.94%  for  percent  coverage.  The  final  split  is 
displayed  on  the  right-hand  side  of  the  figure.  It  is  represented  by  Class  IV  operating 
radius  >=  69.1  km  and  falls  under  Class  I  operating  times  >=  0.905  hrs.  The  result  is  a 
two  percent  increase  to  the  percent  coverage  of  the  baseline  scenario,  which  corresponds 
to  twenty  percent  of  the  observed  range  of  the  MOE. 
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Figure  42.  Regression  Tree  for  Percent  Mission  Coverage  Indicating  the  Significance 
of  Class  I  Operating  Time  and  Class  IV  Operating  Radius. 


As  in  the  mission  value  model,  we  produced  a  contour  plot  to  demonstrate  how 
Class  I  operating  time  and  Class  IV  operating  radius  relate  together  with  their  effect  on 
percent  mission  coverage.  Figure  43  displays  the  contour  plot  for  Class  IV  operating 
radius  vs.  Class  I  operating  time.  Filled  contour  regions  depict  the  percent  mission 
coverage  as  the  two  factors  vary.  The  region  to  the  right  of  the  first  line  on  the  left 
indicates  percent  mission  coverage  at  or  above  eighty  percent.  We  point  out  that  such 
results  occur  when  both  factors  are  set  below  the  base  scenario  settings.  This  result  may 
be  of  interest  when  assessing  platform  requirements  in  future  studies.  Regions  to  the 
right  of  the  second  line  indicate  percent  coverage  at  or  above  83.7%.  Here  we  see  a 
trade-off  required  for  the  factors  such  that  when  one  UAY  is  operating  at  or  below  its 
base  scenario  setting,  the  other  must  operate  at  levels  above  the  base  setting.  However,  in 
the  case  of  the  Class  I  operating  time,  this  amounts  to  only  a  few  minutes  as  noted  in 
Section  C.5  of  this  chapter. 
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Contour  Plot  for  Coverage 


Figure  43.  Contour  Plot  for  Percent  Mission  Coverage  with  Class  IV  Operating 

Radius  vs.  Class  I  Operating  Time. 

E.  INTERESTING  OBSERVATIONS 

This  section  describes  several  interesting  observations  identified  during  our 
analysis.  We  begin  with  a  comparison  of  the  two  measures  of  effectiveness  used  in  our 
regression  models.  Additionally,  we  discuss  the  most  significant  factor  effects  for  each 
measure  of  effectiveness  generated  by  the  ASC-U  version  investigated  in  our  study. 
Next,  we  use  a  parallel  plot  to  highlight  the  importance  of  reviewing  more  than  one  or 
two  MOEs.  Finally,  we  interpret  the  signs  on  the  model  coefficients. 


1.  Measure  of  Effectiveness  Comparison 

We  developed  two  regression  models  using  mission  value  and  percent  mission 
coverage.  Our  intent  was  to  investigate  how  the  UAV  performance  characteristics 
influenced  ASC-U’ s  solution.  After  conducting  regression  analysis  for  our  mission  value 
model,  we  realized  it  was  difficult  to  get  a  tactical  sense  of  the  MOE.  Upon  conducting 
regression  analysis  for  percent  mission  coverage,  we  immediately  recognized  a  difference 
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in  the  two  measures  based  on  their  opposing  significant  factors.  The  switch  between 
Class  IV  operating  radius  and  Class  I  operating  time  demonstrates  a  fundamental 
difference  between  the  two. 

Initially,  mission  value  and  percent  mission  coverage  seem  to  imply  a  similar 
measure  of  effectiveness.  Recall  from  Chapter  II  that  mission  value  is  derived  from  a 
value  rate  that  is  dependent  on  the  mission  area  and  required  sensor  package.  This 
“weighting”  system  may  result  in  a  scenario  with  a  mission  value  that  does  not 
correspond  to  an  equivalent  percent  mission  coverage.  For  example,  mission  areas  with 
more  valuable  sensor  package  requirements  may  be  served  in  such  a  way  to  limit  multiple 
servicing  of  other  mission  areas.  The  cumulative  effect  results  in  a  high  mission  value, 
but  relatively  low  percent  mission  coverage. 

Figure  44  displays  the  difference  between  the  two  measures  by  mapping  the 
combined  514  runs  from  our  initial  and  final  DOEs.  The  green  plot  represents  the  257 
design  points  from  the  initial  DOE  where  we  varied  the  optimization  interval  from  one  to 
ten  simulation  hours.  The  blue  plot  represents  our  final  DOE  where  optimization  interval 
remained  fixed  at  one  simulation  hour.  The  red  plots  indicate  the  corresponding  percent 
mission  coverage  for  each  design  point.  Notice  that  mission  value  is  “over 
representative”  of  the  overall  percent  coverage  across  most  of  the  initial  design. 
However,  the  mission  values  are  more  representative  of  the  percent  coverages  in  the  final 
design.  We  believe  that  the  larger  optimization  intervals  drive  the  allocation  of  UAV 
assets  to  high  value  areas  and  reduce  the  cumulative  effect  of  covering  more  missions 
when  optimization  intervals  are  small. 

After  exploring  the  MOEs  used  in  our  two  regression  models,  we  turned  our 
attention  towards  the  significant  factors  that  affect  a  change  in  the  mean  value  generated 
by  our  design  runs  for  each  MOE.  We  employed  JMP’s  partitioning  platform  in  order  to 
identify  the  regressor  involved  in  the  first  split  against  each  MOE  as  the  response 
variable.  The  results  identified  the  factor  responsible  for  the  optimum  split,  separating 
the  response  variable  into  two  groups  of  different  means.  We  identified  the  significant 
factor,  the  level  of  the  factor  identifying  the  branch  with  the  increased  mean,  the  percent 
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increase  over  the  range  of  the  examined  MOE,  and  the  percent  increase  in  coverage 
(except  in  the  case  of  mission  value).  Table  13  displays  the  results  of  the  exploration. 
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Figure  44.  Comparison  of  Mission  Value  and  Percent  Mission  Coverage  for  Initial 

and  Final  Design  of  Experiments. 

The  most  significant  increase  in  percent  coverage  occurred  for  the  meteorological 
sensor  package  at  15.12%  increase  to  the  mean  when  Class  III  operating  radius  is  set  at 
greater  than  or  equal  to  41.9  km.  This  result  corresponds  to  a  30.24%  increase  over  the 
entire  range  of  percent  coverage  for  meteorological  missions.  While  the  other  splits  did 
not  cause  a  substantial  increase  to  the  mean  of  the  other  MOEs,  we  note  that  five  splits 
resulted  in  a  15%  or  greater  increase  over  the  range  of  the  MOE  studied.  Class  IV 
operating  radius  and  Class  I  operating  time  with  seven  splits  each  represented  the  largest 
number  of  splits  for  the  twenty-one  MOEs.  Class  III  operating  radius  produced  one  split. 
It  was  the  most  significant  factor  to  influence  the  mean  of  meteorological  percent 
coverage.  Interestingly,  Class  I  operating  time  was  the  most  significant  factor  for  percent 
coverage  by  type  of  all  the  UAVs.  Finally,  supply  delivery  missions  were  never  allocated 
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UAYs.  While  this  is  interesting,  there  were  only  five  such  missions  in  the  entire 
scenario. 


MOE 

(Response) 

Significant  Factor 
(First  Split) 

Base  Scenario  Setting 

Partition  Value 
Creating  Positive 
Increase  on  Mean 

Percent  Increase  over 
Range  of  MOE 

Percent  Increase  in 
Coverage 

Value 

4Radius 

75  km 

>=  68.9  km 

8.21% 

Percent  Coverage 

lOprT 

0.830  hrs 

>=  0.787  hr 

8.28% 

0.82% 

Supply  Delivery 

Meterological 

3  Radius 

40  km 

>=  41.9  km 

30.24% 

15.12% 

COMINT 

4Radius 

75  km 

>=  68.9  km 

19.59% 

6.60%  ! 

Weapon 

EOprT 

36  hrs 

>=  38.8  hr 

15.54% 

3.33% 

ELINT 

4Radius 

75  km 

>=  68.9  km 

15.43% 

3.82% 

Class  II UAV 

lOprT 

0.830  hrs 

>=0.816  hr 

14.36% 

0.50% 

Clas  III  UAV 

lOprT 

0.830  hrs 

>=  0.894  hr 

13.00% 

0.57% 

Class  I  UAV 

lOprT 

0.830  hrs 

>=0.816  hr 

11.31% 

1.06% 

Class  IV  UAV 

lOprT 

0.830  hrs 

>=  0.789  hr 

10.38% 

0.21% 

ERMP 

lOprT 

0.830  hrs 

>=  0.787  hr 

7.35% 

0.16% 

GPS  Designator 

EHoriz 

36  hrs 

<30.7  hr 

29.16% 

6.11% 

Laser  Designator 

4Radius 

75  km 

>=  77.6  km 

15.01% 

1.70% 

EO/IR/LR 

lOprT 

0.830  hrs 

>=  0.770  hr 

9.41% 

1.02% 

SAR/MTI 

4Radius 

75  km 

>=  67.6  km 

7.29% 

0.65%  | 

Comms  Relay 

EHoriz 

36  hrs 

<36.7  hr 

6.85% 

0.60% 

EW 

EOprT 

36  hrs 

>=  32.3  hr 

6.35% 

1.37% 

Mine  Detection 

4Radius 

75  km 

>=  66.2  km 

5.90% 

0.61%  [ 

CBRNE 

4Radius 

75  km 

>=  64.2  km 

5.27% 

1.47% 

FOPEN/LIDAR 

EOprT 

36  hrs 

<  42.9  hr 

1.63% 

0.03% 

Table  13.  First  Split  on  Partition  of  MOEs  Indicating  Significant  Factor  and  Percent 
Increase  over  Range  of  MOE  and  Percent  Increase  in  MOE  Coverage. 


Figure  45  displays  a  parallel  plot  to  highlight  the  importance  of  reviewing  more 
than  one  or  two  MOEs.  The  plot  allows  us  to  explore  the  relative  results  of  all  of  the 
MOEs  simultaneously.  We  have  selected  the  six  design  points  that  resulted  in  extreme 
values  as  previously  noted  in  Figure  24  (points  11,  14,  43,  85,  and  253  in  green  and  point 
140  in  red).  We  note  that  while  the  design  points  in  green  represent  the  highest  mission 
values  within  our  DOE,  they  result  in  some  of  the  lowest  percent  coverages  for  numerous 
MOEs.  Regions  1  and  2  emphasize  this  fact  by  indicating  where  designs  85  and  253  are 
separated  from  designs  11,  14,  and  43  within  the  bimodal  distribution  of  meteorological 
percent  coverage  (Meteo).  Additionally,  we  observe  design  point  43  representing  one  of 
the  lowest  outcomes  for  FOPEN/LIDAR  percent  coverage  (FOPEN)  in  region  4. 
Alternatively,  design  point  140  resulted  in  the  lowest  mission  value  out  of  272  runs. 
However,  it  does  not  represent  the  lowest  percent  coverage  for  several  MOEs.  This  is 
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evident  in  region  3  for  FOPEN  coverage.  Again,  we  point  out  that  all  design  points 
resulted  in  0  percent  coverage  for  supply  delivery  (Supply),  as  indicated  in  region  5. 

The  next  section  provides  an  interpretation  of  the  signs  on  the  model  coefficients. 
As  previously  noted  in  the  chapter,  the  time  horizon  factor  influenced  the  response 
variable  counter  to  our  expectations  for  four  of  the  five  UAVs.  We  discuss  additional 
factors  that  have  opposite  signs  than  predicted. 


2.  Interpretation  of  Model  Coefficients 

We  had  expectations  on  how  UAV  performance  capabilities  would  influence  the 
solution  generated  by  ASC-U.  However,  several  of  the  signs  on  the  coefficients  within 
the  models  are  counterintuitive.  Table  14  provides  a  comparison  between  our  predicted 
coefficient  signs  and  the  actual  signs  generated  within  the  regression  models  for  each 
UAY  type.  Factors  with  counterintuitive  signs  are  highlighted  in  yellow  and  bold  type. 
It  is  important  to  note  that  the  difference  between  the  signs  is  not  due  to  multicollinearity 
between  the  regressors;  our  design  methodology  eliminates  this  as  a  possibility.  In 
addition  to  reviewing  the  pairwise  correlations  between  the  factors  of  our  design  matrix 
(see  Chapter  IV,  Section  B.2),  we  also  examined  the  variance  inflation  factors  (VIF) 
between  regressors  within  our  final  models.  The  magnitude  of  the  VIF  serves  as  an 
important  multicollinearity  diagnostic.  The  minimum  and  maximum  VIF  values  within 
our  models  were  1.00  and  1.01,  respectively.  VIF  values  greater  than  5  indicate  serious 
multicollinearity  problems  between  regressors  (Montgomery  et  al.,  2001). 

In  Table  14,  we  see  that  all  of  the  time  horizons  have  a  negative  slope  except  for 
Class  I  UAVs.  We  provided  our  interpretation  of  the  counterintuitive  signs  for  ERMP, 
Class  I,  and  Class  IV  UAVs  in  Sections  C.6  and  D.2  of  this  chapter.  We  offer  the  same 
explanation  for  Class  II  and  III  UAVs.  We  contend  that  the  UAVs  with  greater  range  and 
endurance  seek  out  higher  value  missions  and  miss  the  cumulative  effect  of  serving 
several  “closer- in”  missions.  On  the  other  hand,  Class  I  UAVs  are  the  only  type  of  UAV 
with  time  horizon  interval  less  than  one  simulation  hour.  They  simply  do  not  have  the 
“opportunity”  to  compare  additional  missions. 
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Figure  45.  Parallel  Plot  Displaying  Corresponding  Results  for  MOEs. 
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Table  14  also  displays  additional,  counterintuitive  signs  for  the  coefficients  of  the 
ERMP  UAY’s  operating  radius  and  the  Class  II  UAY’s  operating  and  transition  times. 
The  slopes  suggest  that  smaller  ERMP  operating  radii  and  Class  II  operating  times,  as 
well  as  longer  Class  II  transition  times  increase  mission  value  and  percent  mission 
coverages.  While  these  regressors  are  not  found  in  the  final  models  for  mission  value  or 
percent  mission  coverage,  they  are  consistently  represented  in  this  way  throughout  our 
exploratory  regression  analysis  as  we  constructed  our  final  models.  We  do  not  provide 
an  in-depth  analysis  of  these  counterintuitive  results,  except  to  note  that  their  VIF  values 
fall  within  the  range  described  above.  We  recommend  further  analysis  to  identify  the 
processes  within  the  simulation  model  that  account  for  these  unique  results. 


UAV  Types 

Factors 

Predicted  Sign 
on  Coefficient 

Actual  Sign 
on  Coefficient 

ERMP 

Time  Horizon  (hr) 

+ 

- 

Operating  Time  (hr) 

+ 

+ 

Air  Speed  (km/hr) 

+ 

+ 

Operating  Radius  (km) 

+ 

- 

Transition  Time  (hr) 

- 

- 

Class  I 

Time  Horizon  (hr) 

+ 

+ 

Operating  Time  (hr) 

+ 

+ 

Air  Speed  (km/hr) 

+ 

+ 

Operating  Radius  (km) 

+ 

+ 

Transition  Time  (hr) 

- 

- 

Class  II 

Time  Horizon  (hr) 

+ 

- 

Operating  Time  (hr) 

+ 

- 

Air  Speed  (km/hr) 

+ 

+ 

Operating  Radius  (km) 

+ 

+ 

Transition  Time  (hr) 

- 

+ 

Class  III 

Time  Horizon  (hr) 

+ 

- 

Operating  Time  (hr) 

+ 

+ 

Air  Speed  (km/hr) 

+ 

+ 

Operating  Radius  (km) 

+ 

+ 

Transition  Time  (hr) 

- 

- 

Class  IV 

Time  Horizon  (hr) 

+ 

- 

Operating  Time  (hr) 

+ 

+ 

Air  Speed  (km/hr) 

+ 

+ 

Operating  Radius  (km) 

+ 

+ 

Transition  Time  (hr) 

- 

- 

Table  14.  Comparison  of  Predicted  and  Actual  Coefficient  Signs. 
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VI.  CONCLUSIONS 


If  the  Division  Commanders  want  a  UAV  at  their  level  and  have 

nothing  now,  we  ought  to  give  it  to  them. 

LTG  Scott  Wallace,  CG  Y  Corps  (2003) 
(Sinclair,  2005) 

The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center 
(TRAC)  and  the  Modeling,  Virtual  Environments,  and  Simulations  Institute  (MOVES)  at 
the  Naval  Postgraduate  School,  Monterey,  California  developed  the  Assignment 
Scheduling  Capability  for  UAVs  (ASC-U)  simulation  tool  to  assist  in  the  analysis  of 
UAV  requirements.  TRAC  selected  ASC-U  for  the  Army-wide  UAV  Mix  Analysis  that 
supports  investment  strategies  involving  technology  for  current  operations,  transitioning 
to  modular  forces,  and  developing  and  fielding  the  Future  Combat  Systems  (FCS) 
(Witsken,  2004).  This  thesis  has  explored  the  effects  of  twenty-six  simulation  and  UAV 
factors  on  the  Measures  of  Effectiveness  (MOE)  generated  by  ASC-U. 

We  combined  data  analysis,  exploratory  modeling  and  a  Nearly  Orthogonal  Latin 
Hypercube  design  to  examine  514  design  points.  In  doing  so,  we  expanded  the  use  of 
ASC-U  from  running  single  scenarios  with  baseline  settings  to  a  full  exploration  process 
across  multiple  factors.  The  application  and  insights  gained  will  support  current  and 
future  warfighters.  The  conclusions  suggest  the  following: 

•  The  optimization  interval  significantly  influences  the  solution. 

•  Mission  value  and  percent  mission  coverage  are  not  equivalent  MOEs. 

•  Optimization  interval  of  one  simulation  hour  provides  better  correlation 

between  mission  value  and  percent  mission  coverage. 

•  Each  measure  of  effectiveness  has  a  different  set  of  significant  factors. 

•  Counterintuitive  time  horizon  coefficients  highlight  models’  allocation 

process. 

•  Class  I  operating  time  and  Class  IV  operating  radius  are  main  factors 

influencing  14  of  21  MOEs. 
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•  Class  IV  operating  radius  begins  to  make  significant  improvement  to  percent 

mission  coverage  at  below  baseline  settings. 

•  Longer  Class  I  operating  times  increase  all  other  UAV  mission  percent 

coverages. 

•  State-of-the-art  Design  of  Experiment  and  data  mining  techniques  yield 

insights  impossible  to  gain  otherwise. 

A.  IMMEDIATE  IMPACT 

The  Army  Chief  of  Staff  tasked  TRADOC  to  evaluate  the  current,  modular,  and 
Future  Combat  Systems  (FCS)  force  options  with  regard  to  UAVs  and  develop  an 
investment  strategy.  The  analysis  supports  the  development  of  an  acquisition  strategy  for 
UAVs  at  the  Division  and  Brigade  level.  Additionally,  the  analysis  assists  in  defining  the 
near  term  (2008)  UAV  force  structure  and  the  investment  strategy  for  the  mid  term 
(2013),  and  far  term  (2018). 

This  thesis  addresses  the  directives  of  TRAC  by  providing  an  analysis  of  the  time 
horizon  and  optimization  interval  used  in  ASC-U’s  optimization  process,  UAV 
performance  characteristics,  and  the  appropriate  design  of  experiment  tools  used  in  the 
Army  UAV  Mix  Analysis. 


B.  FOLLOW  ON  RESEARCH 

This  thesis  blended  different  techniques  involving  simulation,  optimization,  and 
state-of-the-art  design  of  experiments  to  achieve  substantial  insights  regarding  the 
allocation  of  UAVs  within  an  advanced  assignment  and  scheduling  simulation.  While  we 
made  significant  advancement  in  combining  DOE  and  data  mining  techniques  with  the 
simulation,  we  suggest  the  following  list  for  future  research: 

•  Continue  integration  of  DOE  tools  with  the  simulation  interface. 

•  Study  additional  scenarios  developed  by  TRAC. 

•  Explore  effects  of  optimization  interval  at  levels  below  one  simulation  hour. 

•  More  rigorously  determine  the  range  of  factors  through  the  acquisition 

community. 

•  Study  trade-offs  between  UAV  capabilities  in  more  detail. 
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•  Explore  number  and  type  of  UAVs  at  different  echelons. 

•  Expand  simulation  to  account  for  attrition  and  maintainability. 

•  Expand  factor  ranges  to  explore  possible  curvilinear  effects. 

•  Investigate  MOEs  displaying  bimodal  distributions. 

•  Conduct  cost  analysis  on  UAV  performance  characteristics  in  relation  to  the 

measures  of  effectiveness  generated  by  the  simulation. 

•  Establish  parameters  representing  mission  area  densities  and  study  their 

effects  on  the  simulation’s  solution. 
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APPENDIX  A.  MISSION  VALUE  MODEL 


The  figure  below  displays  the  statistical  report  for  the  final  regression  model  for 
mission  value. 
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Analysis  of  Variance 


Source 

OF 

Sum  o1  Squares 

Mean  Square 

F  Ratio 

Model 

11 

1 ,45237e1 2 

1 . 320  Sel  1 

2754752 

Error 

250 

1 .2455e^1 1 

479459555 

Prob  ^  F 

C.  Total 

271 

1.57753e12 

<.0001 

Parameter  Estimates 


Term 

Eatimate 

Std  Error 

t  Ratio 

Pro  bHfl 

VIF 

Intercept 

3445141 .3 

34710.67 

99.23 

<.0001 

4  Radius 

6397.9399 

153.1702 

41.77 

<5001 

1.0019502 

lOprT 

311592.55 

13916.5 

22.39 

<5001 

1.0009492: 

EHoriz 

-5344.015 

317.3051 

-16.32 

<5001 

1.0015535 

EOprT 

3934.033 

321.3805 

12.40 

<5  001 

1.001416 

(EHo  r[z-35.9406)±(EO  p  rT-35.9651 ) 

451 .24159 

73.35855 

6.15 

<5001 

1.0115635 

3  Radius 

23487E94 

236.3274 

3.20 

<5001 

15019254 

IRadiua 

1 1  030.025 

1 42-3.31 5 

7.75 

<5001 

1.0023671 

(4  Rad  ius-7550 07)*(4  Rad  ius-75, 0  0  07) 

-135.175 

19.99539 

-9.31 

<5001 

1.0109111 

4Hcriz 

-12354.99 

1902.569 

-6.49 

<5001 

15  006291 

3Hdnz 

-1 0635.5 

1392.326 

-5.65 

<5001 

1.002233 

40prT 

82321434 

192:3,914 

4.23 

<5001 

1.0003135 

Effect  Tests 

Source 

Nparrn 

OF 

Sum  of  Squares 

F  Ratio 

Prob>F 

4Radiua 

1 

1 

•B.36548e1 1 

1744.773 

<.0001 

lOprT 

1 

1 

2:.40362e1 1 

501.3139 

<  0001 

EHoriz 

1 

1 

1.3  5571  ell 

232.7573 

<0001 

EOprT 

1 

1 

7. 353  36e1  0 

153,6305 

<  0001 

EHoriz*EOprT 

1 

1 

1.31 364e1  0 

37.3267 

<  0001 

SRadius 

1 

1 

3.226  37e1  0 

67.2913 

<0001 

IRadiua 

1 

1 

2  88325e1  0 

60.1353 

<0001 

4  Rad  ius*4  Radius 

1 

1 

4. 1 5  5  32e1  0 

35.5557 

<0001 

4Horiz 

1 

1 

2.021 39e10 

42.1701 

<0001 

3Horiz 

1 

1 

1. 5291  lei  0 

31 .392:4 

<0001 

40prT 

1 

1 

3773224540 

13.3936 

<0001 
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APPENDIX  B.  PERCENT  MISSION  COVERAGE  MODEL 


The  figure  below  displays  the  statistical  report  for  the  final  regression  model  for 
percent  mission  coverage. 
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Analysis  of  Variance 


Source 

OF 

Sum  o1  Squares 

Mean  Square 

F  Ratio 

Model 

11 

0  0727T061 

0.006616 

262.3741 

Error 

250 

{1,00609133 

0.000023 

Prob  >  F 

C.  Total 

271 

0.07666194 

<.0001 

Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Pro  bHfl 

VIF 

Intercept 

06235364 

0.007959 

7835 

<.0001 

lOprT 

0.1321763 

0.003075 

42.97 

<.0001 

1.0009701 

4Radiua 

0.0008755 

0.00  0034 

25,65 

<.0001 

1.0019702 

1  Radius 

0.0044126 

0.000315 

13.97 

<.0001 

10026656 

EHoriz 

-0.000768 

0.0  0007 

-10.94 

<.0  001 

1.000451 

4Horiz 

-0.003395 

0.00  0421 

-6.07 

<.0001 

1.0013319 

3Hortz 

-0003028 

0.000416 

-7.24 

<.0001 

1.0019146 

EOprT 

0.0004955 

0.00  0071 

6.97 

<.0001 

1.0019149 

30prT 

0  0025562 

0.00  0425 

6.02 

<.0001 

1.0015039 

[4  Rad  ius-75. 0  0  07f  (4  Rad  ius-75. 0  0  07) 

-0.00003 

0.000004 

-6.65 

<.0001 

1.0010327 

IHortz 

0  0158068 

0.003046 

5,19 

<.0001 

1.0020744 

40prT 

0.002:0676 

0.00  0425 

4  66 

<.0001 

1.0006059 

Effect  Tests 

Source 

Nparrn 

OF 

Sum  of  Squares 

F  Ratio 

Prob  >  F 

lOprT 

1 

1 

0.04325044 

1646.057 

<  0001 

4Radiua 

1 

1 

0.01566462 

563.62:31 

<  0001 

1  Radius 

1 

1 

0.00457151 

195.1237 

<.0001 

EHoriz 

1 

1 

0.002:50451 

1-19.7112 

<  0001 

4Horiz 

1 

1 

0.001  52551 

65,1141 

<  0001 

3Hortz 

1 

1 

0.00122764 

52.4066 

<  0001 

EOprT 

1 

1 

0.00113923 

436264 

<  0001 

30prT 

1 

1 

0.00054515 

36.2032 

<  0001 

4RadiusT4Radius 

1 

1 

0.00109372 

45.5975 

<  0001 

IHoriz 

1 

1 

0.0006302:3 

26.9005 

<  0001 

40prT 

1 

1 

0.000  55399 

23  6465 

<  0001 
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APPENDIX  C.  METEOROLOGICAL  REGRESSION  TREE 


The  regression  tree  above  indicates  that  when  Class  III  aircraft  have  an  operating 
radius  of  greater  than  or  equal  to  41.9  km,  the  mean  percent  coverage  for  meteorological 
missions  is  increased  by  16%  for  the  scenario  used  in  this  study.  This  also  corresponds  to 
nearly  a  200%  increase  to  the  meteorological  percent  coverage  of  the  baseline  scenario. 
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APPENDIX  D.  COMINT  REGRESSION  TREE 
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The  regression  tree  above  displays  the  first  three  splits  for  COMINT  percent 
coverage.  The  third  split  on  the  right  is  represented  by  ERMP  time  horizon  >=  36.3  hrs 
and  falls  under  Class  IV  operating  radius  >=  73.4  km.  The  result  is  a  two  percent 
increase  to  COMINT  percent  coverage  of  the  baseline  scenario.  This  corresponds  to  nine 
percent  of  the  variation  of  the  MOE. 


103 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


104 


APPENDIX  E.  JAVA  CODE  FOR  DATA  CONSOLIDATION 


The  Java™  code  within  this  appendix  calculates  percent  mission  coverage  by 
UAY  type  and  consolidates  the  results.  Microsoft®  Visual  Basic®  macros  were  also 
created  in  order  to  generated  ASC-U  input  fdes,  calculate  MOEs  and  consolidate  the 
results.  The  author  will  provide  additional  code  upon  request. 

/* 

*  File:  CoverageByType . j ava 

*  Created  on  May  1,  2006,  11:34  PM 
*/ 


/*  A 

*  Oauthor  Christopher  J.  Nannini 

*  Class  to  Consolidate  Percent  Coverage  by  UAV  for  257  Design  Points 

*  from  ASC-U  Write-Output  Microsoft  Access  Database. 

*/ 


package  ascuManager; 
import  j ava . sql . Statement ; 
import  j ava . sql . Connection; 
import  j ava . sql . DriverManager ; 
import  j ava . sql . ResultSet ; 
import  j ava . text . DecimalFormat ; 

public  class  CoverageByType  { 

public  static  void  main (String [ ]  args)  { 

DecimalFormat  numberFormat  =  new  DecimalFormat ( " 0 . 00 ") ; 

try  { 

//  set-up  driver  for  database  connection 
Class . f orName ( "sun . j  dbc . odbc . JdbcOdbcDriver " ) ; 

//  identifies  Microsoft  Access  database  file 
System . out . println ( "Design,  "  +  "Class  I,  "  + 

"Class  II,  "  +  "Class  III,  "  +  "Class  IV,  "  + 

"ERMP" ) ; 

for  (int  fileNumber  =  1;  f ileNumber<258 ;  f ileNumber++)  { 
String  filename  =  "C : /Documents  and  Settings/"  + 
"Christopher  Nannini/Desktop/  + 

OPTINT  DATA  COMPLETE/"  + 

"ASCU"  +  Integer . toString ( fileNumber )  +  ".mdb"; 

String  database  =  "j dbc : odbc : Driver^ {Microsoft  Access  + 
Driver ( * . mdb) } ; DBQ=" ; 
database+=  f ilename . trim ( ) + 

" ; Driver ID=22 ; READONLY=true } " ; 

//  creates  a  connection  to  the  database 
Connection  con  =  DriverManager . getConnection ( 
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} 


database  ,"",""); 

//  creates  a  j ava . sql . Statement  to  run  queries 
Statement  s  =  con . createStatement ( ) ; 
double  totalOpenTime  =  0.0; 
double  totalCoveredTime  =  0.0; 
double  percentCovered  =  0.0; 

String []  platform  =  {"Class  I",  "Class  II", 

"Class  III",  "Class  IV",  "ERMP" } ; 

System. out . print ( "ASCU"  + 

Integer . toString ( fileNumber) ) ; 
for  (int  i  =  0;  i<5;  iff)  { 

//  build  Strings  to  interact  with  the  database 
String  columns  =  "Mission,  MissionPlatf ormType,  "  + 
"openTime,  coveredTime  "; 

String  table  =  "  FROM  CoverageByType  "; 

String  filter  =  "  WHERE  MissionPlatf ormType= ' "  + 
platform [i]  + 

String  query  =  "SELECT  "  +  columns  +  table  + 
filter; 

ResultSet  rs  =  s . executeQuery (query) ; 

/ /  iterates  through  the  ResultSet  and  calculates 
totals 

while  (rs . next ( ) )  { 

String  openTime  =  rs . getString ( "openTime" ) ; 
String  coveredTime  = 

rs . getString ( "coveredTime" ) ; 

totalOpenTime  =  totalOpenTime  + 

Double . parseDouble (openTime) ; 
totalCoveredTime  =  totalCoveredTime  + 

Double . parseDouble (coveredTime) ; 

} 

if  (totalOpenTime  ==  0)  { 

System . out . print ("  ,  "); 

System. out .print ("0.00")  ; 

} 

else  { 

System. out .print ("  ,  "); 

System . out . print (number Format . format ( 

100* ( totalCoveredTime/totalOpenTime) ) ) ; 

} 

} 

//  close  statement  and  connection  to  database 
s . close ( ) ; 
con . close ( ) ; 

System . out . print In (""  )  ; 

} 

}  catch  (Exception  err)  { 

System . out . println ( "ERROR :  "  +  err)  ; 

} 
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